If you're moving an LLM from a cloud API to your own server, the first thing to calculate is VRAM, not which model scores best.
The two VRAM consumers
1. Model weights: parameters × bytes per parameter.
| Precision | Bytes per parameter | 70B model |
|---|---|---|
| FP16/BF16 | 2 | ~140GB |
| 8-bit | 1 | ~70GB |
| 4-bit | 0.5 | ~35GB |
2. KV cache: grows with context length, batch size and layer count. Llama 3.3 70B uses about 2.6GB for one 8K-token conversation. Ten concurrent users add roughly 26GB on top of the weights, so the total is around 97GB.
How the three families compare
Llama (dense): predictable scaling. The 8B fits on a 24GB card, while the 70B needs 2x A100 80GB or a single H200.
DeepSeek V3/R1 (MoE, 671B): only about 37-40B parameters are active per token, but every parameter must be in memory. At FP8 that is about 746GB, so plan for an 8x H200/B200 node.
Qwen3: the widest range. The 8B model runs at 4-bit on modest hardware. The Max variant needs a multi-node cluster, and tensor parallelism has to divide the number of attention heads evenly. For example, TP8 with PP3 works where a flat TP24 would fail at startup.
The budget option
R1-Distill-Llama-70B brings DeepSeek-style reasoning to a dense 70B model that fits in 40-48GB of VRAM.
Don't forget the rest of the box
- Storage: NVMe (Gen4/Gen5) to cut model load times
- System RAM: 1.5-2x total VRAM
- PCIe lanes: enough for every GPU to run at x16
- Networking: 100-400Gbps InfiniBand for multi-node clusters
- Power and cooling: redundant PSUs and high-airflow chassis
Full guide with the quick-reference table and hardware recommendations: Qwen vs DeepSeek vs Llama Hardware Requirements
