Zesty automates GPU sharing for AI workloads on Kubernetes

For most AI workloads, a graphics processor, or GPU, is still handed out whole: one application, one chip, whatever the chip is actually doing. The announcement starts from a figure: 66% of organizations hosting generative AI models run some or all of their inference on Kubernetes, the open-source system that schedules containers across a group of machines. Inference is the stage where a trained model answers requests, and GPU capacity for it is, in the announcement's words, expensive and often constrained.

Teams have tried to share these chips. The announcement names time-slicing, NVIDIA's Multi-Process Service (MPS), and Multi-Instance GPU (MIG), which divides each GPU into fixed hardware-based partitions. Its diagnosis is that such allocations cannot keep pace with AI demand that rises and falls. Memory and compute end up stranded, and teams buy more GPUs even while existing capacity sits unused.

NVIDIA's recent open-source release of Run:ai brings GPU fractioning to Kubernetes through a GPU Fractioning Operator, which enforces a memory limit for each workload and works with Kubernetes schedulers. But fractioning only defines how a GPU can be divided. It does not decide who gets what, and the announcement stresses that this requires a separate decision layer. Zesty's Automated Fractional GPUs fills that gap: it continuously determines which workloads share each GPU, how many can safely run together, how much memory each receives, which sharing mode fits, and when allocations should grow or shrink. It draws on measured memory and utilization, observed behavior over time, and the latency and resource limits teams set beforehand, and every change stays within those policies.

The layer deploys as a lightweight agent inside the cluster, alongside NVIDIA's operator and the Kubernetes scheduler, and supports time-slicing and MPS. Nothing in the application or the model needs to change, and installation does not restart running workloads. On a supported GPU stack, Zesty says it manages the Kubernetes configuration itself, without manual manifest edits.

Platform teams that already run inference on Kubernetes are the ones this addresses, and they can use it now, since the product is available as part of the Zesty platform. According to the announcement, the layer makes it possible to get fewer idle chips, run more workloads on hardware already owned, buy less spare capacity for peaks, and lower infrastructure costs while application performance is protected. The announcement quantifies none of it. It gives no utilization gain, no savings percentage, no price and no customer. These remain the vendor's claims, and the test it leaves open is the one a team can run itself: measuring utilization on its own cluster before and after.