LLM VRAM calculator: how much GPU memory a model needs.
Pick a model, a weight format, a context length and how many requests you serve at once. The calculator adds the weights, the KV cache for that model's attention scheme and a 10% margin, then lists the GPU nodes that hold it, cheapest first, with the price per hour.
Weights + KV cache + margin = the GPU memory to rent.
Llama 3.3 70B · 4-bit
- Weights35.3 GB
- KV cache10.7 GB
- Margin, 10%5.11 GB
- Cheapest node1× A100 PCIe 80GB · $0.09/h
- 1× A100 PCIe 80GB80 GB · $0.09/h spot
- 4× RTX A400064 GB · $0.12/h spot
- 2× A4096 GB · $0.12/h spot
- 4× T464 GB · $0.16/h spot
- 2× RTX PRO 4500 Blackwell64 GB · $0.18/h spot
Memory for one node, weights split across its GPUs with tensor parallelism. Full sizing for Llama 3.3 70B.
VRAM for the weights, by model size and format.
Parameters × bytes per parameter, before the KV cache and the margin. Mixtures of experts count their total parameters: every expert is loaded.
| Parameters | BF16 / FP16 | FP8 | 4-bit | Smallest single GPU at 4-bit |
|---|---|---|---|---|
| 7B | 14.0 GB | 7 GB | 3.5 GB | RTX 3070 · 8 GB |
| 8B | 16.0 GB | 8 GB | 4 GB | RTX 3070 · 8 GB |
| 14B | 28.0 GB | 14.0 GB | 7 GB | RTX 3070 · 8 GB |
| 24B | 48.0 GB | 24.0 GB | 12.0 GB | RTX A4000 · 16 GB |
| 32B | 64.0 GB | 32.0 GB | 16.0 GB | RTX 4000 Ada · 20 GB |
| 70B | 140 GB | 70.0 GB | 35.0 GB | A100 40GB · 40 GB |
| 120B | 240 GB | 120 GB | 60.0 GB | A100 PCIe 80GB · 80 GB |
| 235B | 470 GB | 235 GB | 118 GB | H200 NVL · 141 GB |
| 405B | 810 GB | 405 GB | 203 GB | MI325X · 256 GB |
| 671B | 1.34 TB | 671 GB | 336 GB | needs several GPUs |
Weights only, with a 10% margin for the single-GPU column. A long context or many concurrent requests can need a larger card than this column suggests — that is what the calculator above is for.
What the calculator counts, and what it leaves out.
- Weights
- Parameters on disk × 2, 1 or 0.5 bytesFor a model published in a low-bit format, the real checkpoint size
- KV cache
- Per layer: 2 × key-value heads × head dimension × 2 bytes, per tokenSliding-window layers keep only the window; latent attention keeps one compressed vector; hybrid models keep a cache in their full-attention layers only
- Margin
- 10% of the node's memory kept freeActivations, the CUDA context and the serving engine's own buffers
- Format support
- FP8 needs Hopper, Ada, Blackwell or MI300-class GPUsFP4+FP8 checkpoints need Blackwell; 4-bit and MXFP4 run from Ampere onwards
- Not counted
- SpeedTokens per second follow memory bandwidth and active parameters; we do not publish throughput we have not measured
Sized models
64 pagesEvery built-in model has a page with the memory at each context length, every GPU that runs it and the price per hour:
VRAM, explained
How much VRAM do I need per billion parameters?
For the weights: 2 GB per billion parameters at BF16 or FP16, 1 GB at FP8 and about 0.5 GB at 4-bit. A 70B model is about 141 GB at BF16 and 35 GB at 4-bit. Add the KV cache for your context and number of concurrent requests, and keep roughly 10% of the GPU memory free.
How is the KV cache calculated?
For each layer that keeps a cache: 2 (keys and values) × key-value heads × head dimension × bytes per value, per token, multiplied by context length and by the number of sequences served at once. Layers with a sliding window only keep the window; latent-attention models (DeepSeek, Kimi, GLM-5) keep one compressed vector per token; hybrid models (Qwen3.5 to 3.8, Nemotron 3) keep a cache in a minority of their layers. The calculator applies the right rule for each model from its published configuration.
Does a mixture-of-experts model need less VRAM?
No: every expert must be loaded, so memory follows the total parameter count. The active parameters decide how much of the weights is read per token, which drives speed, not memory.
Can I split a model across several GPUs?
Yes. Tensor parallelism splits the weights and the cache across the GPUs of one node, so a model that needs 150 GB runs on two 80 GB cards. Nodes here come in 1, 2, 4 or 8 GPUs; the calculator picks the smallest node of each GPU model that holds the total.
Is 4-bit quantisation good enough?
Often, for large models; less reliably for small ones. Quantisation to 4 bits (AWQ, GPTQ, GGUF Q4) quarters the memory at a quality cost that depends on the model and the task. When a model is published in a low-bit format (MXFP4 for gpt-oss, INT4 for Kimi K2.6, FP4+FP8 for DeepSeek V4), that format is what its developers serve, and the calculator uses the real checkpoint size.
See also GPU requirements for 64 open models · every GPU with its memory and price · rent or buy a GPU.
Your first H100 for $0.55 an hour.
Prepaid in crypto, pay as you go: no card, no contract, no minimum commitment. Add credit, launch, stop whenever you want.
Billed per minute from the moment the instance is reachable. Minimum credit $30, no subscription.