New B200 spot capacity is live in US East from $1.69 per GPU-hour. See availability →
Deploy a GPU
Alibaba Qwen · Mixture of experts · released July 2025

Qwen3 235B A22B GPU requirements: 118 GB at 4-bit, 470 GB at BF16

Qwen3 235B A22B needs 470 GB of GPU memory for its weights at BF16, 235 GB at FP8 and 118 GB at 4-bit, plus 6.31 GB of KV cache for a 32K context. The cheapest node that runs it here is 2× A100 PCIe 80GB at 4-bit, for $0.18 an hour on spot.

Architecture read from the model's config.json on Hugging Face, 26 October 2026 · prices from the live catalogue · all 47 GPUs

Cheapest node · 32K context per node-hour, spot
4-bit2× A100 PCIe 80GB · 124 GB needed $0.18$0.38 on-demand
FP88× L40 · 241 GB needed $0.72$1.52 on-demand
BF168× A100 PCIe 80GB · 476 GB needed $0.72$1.52 on-demand
Deploy 2× A100 PCIe All models

Weights plus a 32K 16-bit KV cache, with 10% of the node's memory kept free. Billed per minute.

Parameters
235B22B active per token
Weights, 4-bit
118 GBBF16: 470 GB
KV cache, 32K
6.31 GBOne sequence, 16-bit cache
Context window
256KNative
Memory by format

How much VRAM Qwen3 235B A22B needs, format by format.

Weights are parameters × bytes per parameter. Each context column adds the KV cache of one sequence of that length.

GPUs by memory
Format Weights + 8Kcontext+ 32Kcontext+ 128Kcontext+ 256Kcontext Cheapest node, 32K
BF16Published weights, fine-tuning 470 GB 472 GB476 GB495 GB521 GB 8× A100 PCIe 80GB · $0.72/h
FP8Hopper, Ada, Blackwell, MI300 235 GB 237 GB241 GB260 GB286 GB 8× L40 · $0.72/h
4-bitAWQ, GPTQ, GGUF Q4 118 GB 119 GB124 GB143 GB168 GB 2× A100 PCIe 80GB · $0.18/h
Arithmetic on published facts, not a benchmark.

The serving engine adds its own buffers, and an 8-bit KV cache halves the context columns. We keep 10% of a node's memory free when we say a model fits. Speed is a separate question: it follows memory bandwidth and, for a mixture of experts, the active parameters.

Where it runs

GPUs that run Qwen3 235B A22B, cheapest first.

For each format, the smallest node of each GPU model that holds the weights and a 32K context, sorted by spot price. One row per GPU model; larger nodes of the same GPU work too.

All prices
Format GPU GPU memory Spot On-demand 730 h spot
4-bit 2× A100 PCIe 80GBAmpere · 124 GB needed 160 GB $0.18 $0.38 $131.40 Deploy
4-bit 4× A40Ampere · 124 GB needed 192 GB $0.24 $0.36 $175.20 Deploy
4-bit 4× A100 40GBAmpere · 124 GB needed 160 GB $0.36 $0.68 $262.80 Deploy
4-bit 4× L40Ada Lovelace · 124 GB needed 192 GB $0.36 $0.76 $262.80 Deploy
FP8 8× L40Ada Lovelace · 241 GB needed 384 GB $0.72 $1.52 $525.60 Deploy
FP8 4× RTX PRO 6000 BlackwellBlackwell · 241 GB needed 384 GB $0.76 $1.56 $554.80 Deploy
FP8 8× L40SAda Lovelace · 241 GB needed 384 GB $1.20 $2.00 $876.00 Deploy
FP8 2× H200 NVLHopper · 241 GB needed 282 GB $1.38 $1.98 $1,007.40 Deploy
BF16 8× A100 PCIe 80GBAmpere · 476 GB needed 640 GB $0.72 $1.52 $525.60 Deploy
BF16 8× A100 SXM 80GBAmpere · 476 GB needed 640 GB $1.52 $2.32 $1,109.60 Deploy
BF16 8× RTX PRO 6000 BlackwellBlackwell · 476 GB needed 768 GB $1.52 $3.12 $1,109.60 Deploy
BF16 4× H200 NVLHopper · 476 GB needed 564 GB $2.76 $3.96 $2,014.80 Deploy

Spot capacity can be reclaimed with a 2-minute notice; for a live endpoint, use on-demand. How spot works · serving a language model.

Context and the KV cache

What a long context costs in memory.

Grouped-query attention, 4 KV heads of 128 across 94 layers.

Context KV cache, 1 sequence 8 sequences
8K tokens 1.58 GB 12.6 GB
32K tokens 6.31 GB 50.5 GB
128K tokens 25.2 GB 202 GB
256K tokens 50.5 GB 404 GB

Serving several users

KV cache

Every concurrent request keeps its own cache, so memory for context grows with users × context length. A server batching 8 requests of 32K needs eight times the single-sequence figure — 50.5 GB here.

  • Recommended node2× A100 PCIe 80GB
  • Weights at 4-bit118 GB
  • Context that fits, 1 sequence134K

Paged attention (vLLM, SGLang) allocates the cache in blocks as requests grow, so a node holds more short requests than the worst case suggests.

Model facts

Qwen3 235B A22B, as published.

Developer
Alibaba QwenFlagship Qwen3 mixture of experts, the 2507 checkpoint
Parameters
235B total, 22B active per token
Architecture
Mixture of experts, 94 layersGrouped-query attention, 4 KV heads of 128 across 94 layers
Context window
262,144 tokensAs set in the published configuration
Published format
BF16, 470 GB
Licence
Apache 2.0
Released
21 July 2025
Source
Qwen/Qwen3-235B-A22B-Instruct-2507config.json read 26 October 2026

Serve it with vLLM

template: vLLM

On 2× A100 PCIe 80GB, from the instance terminal:

terminal
$ vllm serve Qwen/Qwen3-235B-A22B-Instruct-2507 \
    --tensor-parallel-size 2 \
    --max-model-len 32768

Very recent architectures need a recent vLLM; the model card gives the minimum version and any extra flags.

FAQ

Qwen3 235B A22B on a GPU — the questions that come up

How much VRAM does Qwen3 235B A22B need?

For the weights alone: 470 GB at BF16, 235 GB at FP8 and 118 GB at 4-bit. Add 6.31 GB of KV cache for one 32K-token context, or 50.5 GB at the full 256K window, and keep about 10% of the memory free for activations.

Can Qwen3 235B A22B run on a single GPU?

One RTX 4090 (24 GB): no. One RTX 5090 (32 GB): no. One H100 SXM (80 GB): no. One H200 SXM (141 GB): yes, at 4-bit. One B200 SXM (180 GB): yes, at 4-bit. Those answers assume a 32K context and the widest format that still fits.

What is the cheapest way to run Qwen3 235B A22B?

Here, 2× A100 PCIe 80GB at 4-bit: $0.18 per hour on spot and $0.38 on-demand, billed per minute — $131.40 for a 730-hour month on spot. That node has 160 GB of GPU memory for the 124 GB this setup needs.

Does Qwen3 235B A22B need less memory because only 22B parameters are active?

No. A mixture-of-experts model routes each token through a few experts, but every expert has to be resident in GPU memory, so the memory bill follows the 235B total. What the 22B active parameters change is the work per token: generation reads far less of the weights each step than a dense model of the same size would, which is why it is fast for its size.

Which format should I run it in?

BF16 keeps the published weights untouched and is the format for fine-tuning. FP8 halves the memory and runs in hardware on Hopper, Ada, Blackwell and MI300-class GPUs. 4-bit quantisation (AWQ, GPTQ, GGUF) quarters it, at a quality cost that is larger on small models than on big ones; check it against your own evaluation before relying on it.

How do I deploy it?

Pick the node in the table above and launch it with the vLLM template; the model downloads from Hugging Face (Qwen/Qwen3-235B-A22B-Instruct-2507) on first start. Spread the weights over the node with --tensor-parallel-size equal to the GPU count, and set --max-model-len to the context you need. New architectures need a recent vLLM: check the model card for the minimum version.

Sizing another model? GPU requirements for 64 open models · serving a language model · fine-tuning.

Get started

Qwen3 235B A22B on 2× A100 PCIe 80GB for $0.18 an hour.

Launch the node with the vLLM template, point it at Qwen/Qwen3-235B-A22B-Instruct-2507, stop it when you are done. Prepaid, billed by the minute.

Deploy 2× A100 PCIe All open models

Billed per minute from the moment the instance is reachable. Minimum credit $30, no subscription.