NVIDIA Run:ai GPU Fractioning on Amazon EKS: Memory Contracts and Deployment Trade-offs
Your GPU is sitting idle while your second pod waits in Pending — and AWS just showed the fix is a memory contract, not a bigger instance.
That's the real story here. NVIDIA Run:ai GPU Fractioning lets two pods share a single A10G on EKS by enforcing per-container GPU-memory limits — 8 GiB contracts each, both running probes that allocate 6 GiB. Without it, the device plugin exposes the GPU as one indivisible nvidia.com/gpu resource, and the second pod just waits. Fractioning turns a scheduling problem into an accounting problem. That is genuinely useful. It is also, emphatically, not a performance win — and the walkthrough never pretends otherwise.
The density win is real, but narrow
Two pods on one A10G where the baseline allowed one. For notebooks with idle periods, short evaluation jobs, and small inference services, that's a legitimate scheduling fix. The enforced memory limit plus per-pod metrics from the metricsd daemon makes the shared arrangement observable, not just hopeful.
A memory contract is not a compute share
Here's the part vendors would rather you skip: the contract caps memory, not compute. Nothing in this walkthrough measures throughput, latency under concurrent load, or whether your two workloads step on each other when both actually work. Density went up. Everything else is unmeasured — and AWS is honest about that.
The fault-isolation fine print
Fractioning runs on MPS, and the error domains depend on the compute mode: separate domains in the default time-slicing mode, a shared one in sm-sharing. Those are descriptions from NVIDIA's docs, not guarantees, and the walkthrough doesn't test fault containment for either. Sharing a GPU means sharing its failure modes. Plan accordingly.
The deployment tax is steep
This is not a Helm chart you throw at a cluster on a Friday. The validated stack needed a custom AL2023 image because the fractioning operator demanded an R615 driver branch that the EKS-optimized images didn't ship, plus pinned KAI Scheduler and GPU Operator versions, a tainted GPU node group, and a Packer-built AMI. Reproducibility depends on exact pins, and the driver version selector picks a branch, not a patch. Budget a day, not an hour.
Go deeper
The full walkthrough — component versions, the NvFractions annotations, the baseline comparison, and the cleanup steps for the billable resources it spins up — is on Agentic Affairs:
Originally published on Agentic Affairs (https://agentic-affairs.com)