EverXCloud (everxcloud.ai) is a US GPU marketplace where you rent NVIDIA GPUs by the hour from reviewed suppliers, with EverXCloud Inc. as merchant of record on every rental. Before renting anything, the most common question we see is simple: will this model fit on that GPU? To answer it, EverXCloud publishes a free LLM VRAM calculator and its source code on GitHub (everxcloud/gpu-vram-calculator). This post explains the formulas it uses, so you can run the numbers yourself or check the tool's output.

The three parts of inference memory

GPU memory for inference has three components:

  1. Weights, which depend on parameter count and precision.
  2. KV cache, which grows with context length and batch size.
  3. Overhead, which covers activations, the CUDA context, and runtime buffers.

The calculator combines them like this:

Inference total (GB) = (weights + KV cache) × (1 + overhead %) 

The default overhead is 10% for language models and 20% for image pipelines.

Weights: parameters times bytes

Weights (GB) = billions of parameters × bytes per parameter 

Bytes per parameter depend on precision:

Precision Bytes per parameter
FP32 4
BF16 / FP16 2
FP8 / INT8 1
INT4 0.5

A 70B model therefore needs 140 GB in BF16, 70 GB in FP8 or INT8, and 35 GB in INT4, before KV cache and overhead. Quantization is the single largest lever you have.

KV cache: the part people forget

During generation, the model stores keys and values for every token in the context. That cache is often what pushes a "fits on paper" model over the limit.

KV cache (GB) = 2 × layers × KV dimension × context tokens × batch × 2 bytes ÷ 10^9 
  • The leading 2 counts keys and values.
  • The trailing 2 bytes assumes keys and values are stored in BF16 or FP16.
  • KV dimension is the number of key/value heads multiplied by the size of one head.

The last point matters. Models with grouped-query attention use far fewer key/value heads than query heads, which shrinks the cache considerably. That is why the calculator's presets read the published configuration of each model (layers, key/value heads, hidden size) instead of guessing from parameter count. For a model that is not in the preset list, choose Custom and copy the values from its configuration file.

Worked examples

All examples below use an 8,192-token context, batch 1, and 10% overhead.

Llama 3.3 70B. About 158 GB in BF16, 81 GB in FP8, and 42 GB in INT4. In practice that means two 80 GB GPUs in BF16, two 80 GB GPUs in FP8 (81 GB is just over one card), and a single 48 GB GPU in INT4.

DeepSeek R1 Distill Qwen 32B on an RTX 4090. Yes, in INT4: the 32B distill needs about 20 GB, within the 24 GB of an RTX 4090. In BF16 it needs about 75 GB, so four 24 GB GPUs or a single GPU with more memory. Longer contexts or larger batches grow the KV cache, so leave headroom.

Fine-tuning needs a different formula

Training stores far more than weights.

Full fine-tuning with AdamW in BF16 mixed precision takes 16 bytes per parameter:

Item Bytes
BF16 weights 2
Gradients 2
FP32 master copy 4
Two Adam moments 8

Activations come on top. The calculator estimates them with gradient checkpointing and fused attention (Korthikanti et al., 2022):

Training activations (GB) = sequence tokens × batch × hidden size × (2 × layers + 34) bytes ÷ 10^9 

plus logits and their gradient, sequence tokens × batch × vocabulary × 8 bytes in FP32.

For Llama 3.1 8B with 2,048-token sequences, a full fine-tune lands around 145 GB.

LoRA keeps the base model frozen at the chosen precision and trains adapters. The calculator assumes adapters on 1% of parameters, a cautious upper bound, at 16 bytes each, plus activations. The same 8B model drops to about 22 GB with BF16 weights, and to about 9 GB with QLoRA (INT4 base).

Image models count the whole pipeline

For FLUX.1 and SDXL, the calculator counts the denoiser, the text encoders, and the VAE for one 1024 × 1024 image, with 20% overhead and no KV cache. The full FLUX.1 [dev] pipeline (transformer, T5-XXL and CLIP text encoders, VAE) needs about 41 GB in BF16 and 20 GB in FP8. Offloading the text encoder once the prompt is encoded brings BF16 within reach of a 32 GB GPU.

Multi-GPU and the limits of an estimate

The rule for multiple cards is straightforward: a model fits on n GPUs when n × the memory of one GPU is at least the total. That assumes the model is split across them with tensor or pipeline parallelism. Interconnect and framework overhead are not counted, so treat multi-GPU results as optimistic.

More generally, the output is a planning estimate. Real usage depends on model architecture, inference engine, quantization metadata, and memory fragmentation. Test on the target GPU before committing to a long rental.

Embed it or fork it

The calculator runs in the reader's browser and can be embedded for free in a blog post, a course, or a documentation page with an iframe (the snippet is on the calculator page). The repository is public if you want to add a model preset or adapt the formulas to your own stack.

Once you know the memory figure, the next step is matching it to hardware. The EverXCloud GPU catalog lists manufacturer specifications for data center, workstation, and GeForce GPUs, and the EverXCloud marketplace shows live offers from reviewed suppliers.