LLM GPU requirements: how much VRAM 64 open models need.

Count the weights first: about 2 GB per billion parameters at BF16, 1 GB at FP8 and 0.5 GB at 4-bit. Then add the KV cache, which grows with context length and with the number of requests served at once, and keep around 10% of the memory free. Llama 3.3 70B, for example, is 141 GB at BF16 and 35.3 GB at 4-bit, plus 10.7 GB of cache for a 32K context.

No. Only a few experts run for each token, but all of them must sit in GPU memory, so the memory bill follows the total parameter count. 39 of the 64 models on this page are mixtures of experts; their active parameter count tells you how fast they generate, not how much memory they need.

During generation, the model keeps the keys and values of every previous token so it does not recompute them. That cache grows linearly with context length and with concurrent requests. Architectures differ a lot: grouped-query attention keeps a few key-value heads, latent attention (DeepSeek, Kimi, GLM-5) keeps one compressed vector per token, and hybrid models (Qwen3.5 to 3.8, Nemotron 3) keep a cache only in a minority of their layers.

Run a model in the format it is published in when there is one (FP8 for DeepSeek and GLM-5.3, MXFP4 for gpt-oss, INT4 for Kimi K2.6 and K3). Otherwise BF16 keeps the weights untouched, FP8 halves the memory and runs in hardware on Hopper, Ada, Blackwell and MI300-class GPUs, and 4-bit quarters it at a quality cost that is larger on small models.

Parameter counts, layers, key-value heads, attention windows and checkpoint sizes are read from each model's config.json and file listing on Hugging Face (26 October 2026); the memory is arithmetic on those facts, and the prices are the live catalogue. We do not publish throughput figures: we have not measured them.