DeepSeek R1 Distill Llama 8B GPU requirements: 4.02 GB at 4-bit, 16.1 GB at BF16
DeepSeek R1 Distill Llama 8B needs 16.1 GB of GPU memory for its weights at BF16, 8.03 GB at FP8 and 4.02 GB at 4-bit, plus 4.29 GB of KV cache for a 32K context. The cheapest node that runs it here is 1× RTX 3080 at 4-bit, for $0.03 an hour on spot.
Weights plus a 32K 16-bit KV cache, with 10% of the node's memory kept free. Billed per minute.
- Parameters
- 8BDense
- Weights, 4-bit
- 4.02 GBBF16: 16.1 GB
- KV cache, 32K
- 4.29 GBOne sequence, 16-bit cache
- Context window
- 128KNative
How much VRAM DeepSeek R1 Distill Llama 8B needs, format by format.
Weights are parameters × bytes per parameter. Each context column adds the KV cache of one sequence of that length.
| Format | Weights | + 8Kcontext | + 32Kcontext | + 128Kcontext | Cheapest node, 32K |
|---|---|---|---|---|---|
| BF16Published weights, fine-tuning | 16.1 GB | 17.1 GB | 20.4 GB | 33.2 GB | 1× A30 · $0.05/h |
| FP8Hopper, Ada, Blackwell, MI300 | 8.03 GB | 9.1 GB | 12.3 GB | 25.2 GB | 1× L4 · $0.05/h |
| 4-bitAWQ, GPTQ, GGUF Q4 | 4.02 GB | 5.09 GB | 8.31 GB | 21.2 GB | 1× RTX 3080 · $0.03/h |
The serving engine adds its own buffers, and an 8-bit KV cache halves the context columns. We keep 10% of a node's memory free when we say a model fits. Speed is a separate question: it follows memory bandwidth and, for a mixture of experts, the active parameters.
GPUs that run DeepSeek R1 Distill Llama 8B, cheapest first.
For each format, the smallest node of each GPU model that holds the weights and a 32K context, sorted by spot price. One row per GPU model; larger nodes of the same GPU work too.
| Format | GPU | GPU memory | Spot | On-demand | 730 h spot | Action |
|---|---|---|---|---|---|---|
| 4-bit | 1× RTX 3080Ampere · 8.31 GB needed | 10 GB | $0.03 | $0.06 | $21.90 | Deploy |
| 4-bit | 1× RTX A4000Ampere · 8.31 GB needed | 16 GB | $0.03 | $0.05 | $21.90 | Deploy |
| 4-bit | 1× RTX 4070Ada Lovelace · 8.31 GB needed | 12 GB | $0.04 | $0.06 | $29.20 | Deploy |
| 4-bit | 1× RTX 3080 TiAmpere · 8.31 GB needed | 12 GB | $0.04 | $0.07 | $29.20 | Deploy |
| FP8 | 1× L4Ada Lovelace · 12.3 GB needed | 24 GB | $0.05 | $0.08 | $36.50 | Deploy |
| FP8 | 1× RTX 5080Blackwell · 12.3 GB needed | 16 GB | $0.06 | $0.09 | $43.80 | Deploy |
| FP8 | 1× RTX 5070 TiBlackwell · 12.3 GB needed | 16 GB | $0.06 | $0.09 | $43.80 | Deploy |
| FP8 | 1× RTX 4080 SUPERAda Lovelace · 12.3 GB needed | 16 GB | $0.06 | $0.09 | $43.80 | Deploy |
| BF16 | 1× A30Ampere · 20.4 GB needed | 24 GB | $0.05 | $0.08 | $36.50 | Deploy |
| BF16 | 1× L4Ada Lovelace · 20.4 GB needed | 24 GB | $0.05 | $0.08 | $36.50 | Deploy |
| BF16 | 1× RTX 3090Ampere · 20.4 GB needed | 24 GB | $0.05 | $0.08 | $36.50 | Deploy |
| BF16 | 1× RTX A5000Ampere · 20.4 GB needed | 24 GB | $0.06 | $0.09 | $43.80 | Deploy |
Spot capacity can be reclaimed with a 2-minute notice; for a live endpoint, use on-demand. How spot works · serving a language model.
What a long context costs in memory.
Grouped-query attention, 8 KV heads of 128 across 32 layers.
| Context | KV cache, 1 sequence | 8 sequences |
|---|---|---|
| 8K tokens | 1.07 GB | 8.59 GB |
| 32K tokens | 4.29 GB | 34.4 GB |
| 128K tokens | 17.2 GB | 137 GB |
Serving several users
KV cacheEvery concurrent request keeps its own cache, so memory for context grows with users × context length. A server batching 8 requests of 32K needs eight times the single-sequence figure — 34.4 GB here.
- Recommended node1× RTX 3080
- Weights at 4-bit4.02 GB
- Context that fits, 1 sequence37K
Paged attention (vLLM, SGLang) allocates the cache in blocks as requests grow, so a node holds more short requests than the worst case suggests.
DeepSeek R1 Distill Llama 8B, as published.
- Developer
- DeepSeekR1 reasoning distilled into Llama 3.1 8B
- Parameters
- 8B
- Architecture
- Dense model, 32 layersGrouped-query attention, 8 KV heads of 128 across 32 layers
- Context window
- 131,072 tokensAs set in the published configuration
- Published format
- BF16, 16.1 GB
- Licence
- MIT
- Released
- 20 January 2025
- Source
- deepseek-ai/DeepSeek-R1-Distill-Llama-8Bconfig.json read 26 October 2026
Serve it with vLLM
template: vLLMOn 1× RTX 3080, from the instance terminal:
$ vllm serve deepseek-ai/DeepSeek-R1-Distill-Llama-8B \ --tensor-parallel-size 1 \ --max-model-len 32768
Very recent architectures need a recent vLLM; the model card gives the minimum version and any extra flags.
DeepSeek R1 Distill Llama 8B on a GPU — the questions that come up
How much VRAM does DeepSeek R1 Distill Llama 8B need?
For the weights alone: 16.1 GB at BF16, 8.03 GB at FP8 and 4.02 GB at 4-bit. Add 4.29 GB of KV cache for one 32K-token context, or 17.2 GB at the full 128K window, and keep about 10% of the memory free for activations.
Can DeepSeek R1 Distill Llama 8B run on a single GPU?
One RTX 4090 (24 GB): yes, at BF16. One RTX 5090 (32 GB): yes, at BF16. One H100 SXM (80 GB): yes, at BF16. One H200 SXM (141 GB): yes, at BF16. One B200 SXM (180 GB): yes, at BF16. Those answers assume a 32K context and the widest format that still fits.
What is the cheapest way to run DeepSeek R1 Distill Llama 8B?
Here, 1× RTX 3080 at 4-bit: $0.03 per hour on spot and $0.06 on-demand, billed per minute — $21.90 for a 730-hour month on spot. That node has 10 GB of GPU memory for the 8.31 GB this setup needs.
How long a context fits on the recommended node?
With the weights at 4-bit on 1× RTX 3080, the remaining memory holds a 16-bit KV cache for about 38,032 tokens of context for one sequence (37K). An 8-bit KV cache doubles that; serving several users at once divides it between them.
Which format should I run it in?
BF16 keeps the published weights untouched and is the format for fine-tuning. FP8 halves the memory and runs in hardware on Hopper, Ada, Blackwell and MI300-class GPUs. 4-bit quantisation (AWQ, GPTQ, GGUF) quarters it, at a quality cost that is larger on small models than on big ones; check it against your own evaluation before relying on it.
How do I deploy it?
Pick the node in the table above and launch it with the vLLM template; the model downloads from Hugging Face (deepseek-ai/DeepSeek-R1-Distill-Llama-8B) on first start. Spread the weights over the node with --tensor-parallel-size equal to the GPU count, and set --max-model-len to the context you need. New architectures need a recent vLLM: check the model card for the minimum version.
Sizing another model? GPU requirements for 64 open models · serving a language model · fine-tuning.
DeepSeek R1 Distill Llama 8B on 1× RTX 3080 for $0.03 an hour.
Launch the node with the vLLM template, point it at deepseek-ai/DeepSeek-R1-Distill-Llama-8B, stop it when you are done. Prepaid, billed by the minute.
Billed per minute from the moment the instance is reachable. Minimum credit $30, no subscription.