Qwen3 235B-A22B
A 235B mixture-of-experts that only activates 22B parameters per token — flagship-class quality with mid-size-model generation speed, if you can find the memory to hold all 235B of it.
Last verified 2026-07 · Page created; Q4_K_M/Q5_K_M/Q8_0 sizes verified against unsloth/Qwen3-235B-A22B-GGUF (summed across split gguf files).
What it is good at and why the MoE split matters
Qwen3-235B-A22B routes each token through 8 of its 128 experts, so only 22B parameters activate per forward pass even though all 235B have to be resident in memory. That split is the whole point of MoE: you pay the VRAM cost of the full 235B, but generation speed tracks the much smaller 22B active count — closer to a 32B-class model's throughput than a dense 235B would be.
The hardware it wants
Q4_K_M is 142 GB — past what a pair of 24 GB GPUs (48 GB) or even a 96 GB quad-24GB rig can hold in one shot without offload. The realistic paths are a multi-GPU workstation with 160 GB+ pooled VRAM (e.g. 2x80GB datacenter cards), or a large unified-memory Mac Studio (192 GB+) where capacity, not bandwidth, is the limiting resource. A 128 GB Strix Halo mini-PC falls just short of Q4_K_M's 142 GB and needs a more aggressive quant (Q3-class) to fit.
Q5_K_M (166.8 GB) and Q8_0 (250 GB) push further into multi-GPU-server or large-unified-memory territory only — there is no single-consumer-GPU story for this model at any quant above the most aggressive compressions.
VRAM by quantization
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| Q8_0 | 250.1 GB | 321.4 GB | 422.7 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q5_K_M | 166.8 GB | 225.6 GB | 326.9 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q4_K_M | 142.1 GB | 197.2 GB | 298.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
Runs on
No catalog GPU in our reference list fits this model yet — check the full AI-scored catalog.
Recommended builds
Three ways to size a machine for Qwen3 235B-A22B, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
Tokens/sec expectations
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~98.3 |
| RTX 4090 | 1008 GB/s | ~55.3 |
| RTX 6000 Ada | 960 GB/s | ~52.7 |
| RX 7900 XTX | 960 GB/s | ~52.7 |
| RTX 5080 | 960 GB/s | ~52.7 |
| RTX 5070 Ti | 896 GB/s | ~49.2 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Build it, or rent the compute?
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Frequently asked questions
Can I run this on a 128 GB Strix Halo mini-PC?
Not at Q4_K_M — that quant is 142 GB, past the 128 GB unified-memory ceiling once you account for the OS and other overhead. A more aggressive quant (Q3_K-class) is needed to fit, with a further quality tradeoff.
Why is a 235B model not just "run it like a 70B but bigger"?
Because it is a mixture-of-experts model: all 235B parameters must be held in memory (that is what drives your VRAM/RAM requirement), but only 22B activate per token, so generation speed behaves closer to a 22-32B dense model. Memory and speed decouple in a way a dense model never does.
What is the realistic minimum hardware for this model?
A multi-GPU setup with 160 GB+ pooled VRAM, or a Mac Studio/similar with 192 GB+ unified memory, for Q4_K_M. Below that, you are into sub-4-bit dynamic quants with a real quality cost.