Qwen3 32B
Alibaba's dense 32B flagship with a switchable "thinking" mode built in — an Apache-licensed model that fits a single 24 GB GPU and holds up against models twice its size.
Last verified 2026-07 · Page created; Q4_K_M/Q5_K_M/Q8_0 sizes verified against Qwen/Qwen3-32B-GGUF; FP16(BF16) size cross-checked against community re-uploads.
What it is good at
Qwen3 32B is Alibaba's dense flagship from the April 2025 Qwen3 family, and its headline feature is a hybrid mode you switch per-request: a slower "thinking" mode for math/code/logic that reasons step by step, and a fast direct-answer mode for everyday chat — one model instead of separate reasoning and non-reasoning checkpoints. It is Apache 2.0 licensed, so no usage restrictions.
Native context is 32K tokens, extendable to 131K with YaRN scaling if your serving stack supports it — plan around the native 32K for normal use and only reach for the extended window on long-document tasks.
The hardware it wants
Q4_K_M is 19.8 GB, fitting a single 24 GB GPU with genuine headroom for context — this is the model this site's "starter AI build" tier is sized around once you outgrow an 8B. Q5_K_M (23.2 GB) is a tight but workable fit on the same 24 GB card at moderate context. Q8_0 (34.8 GB) needs 48 GB or two pooled 24 GB cards.
A single 16 GB GPU can run smaller quants (Q3-class) but loses meaningful quality — 24 GB is the tier where this model is actually comfortable.
VRAM by quantization
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 65.5 GB | 80.0 GB | 94.2 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q8_0 | 34.8 GB | 44.7 GB | 58.9 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Fits comfortably |
| Q5_K_M | 23.2 GB | 31.4 GB | 45.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Fits comfortably | Fits comfortably |
| Q4_K_M | 19.8 GB | 27.5 GB | 41.6 GB | Doesn't fit | Doesn't fit | Tight — reduced context | Fits comfortably | Fits comfortably |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
Runs on
Unified-memory machines — Apple Silicon Mac Studio/MacBook Pro (Max/Ultra chips) and AMD Strix Halo mini-PCs — share system RAM and GPU memory in one pool, so a 64–128 GB configuration can hold models that would otherwise need multiple discrete GPUs. Throughput is bandwidth-bound like any GPU; it isn’t modeled in the table above yet.
Recommended builds
Three ways to size a machine for Qwen3 32B, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
Tokens/sec expectations
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~63.4 |
| RTX 4090 | 1008 GB/s | ~35.6 |
| RTX 6000 Ada | 960 GB/s | ~33.9 |
| RX 7900 XTX | 960 GB/s | ~33.9 |
| RTX 5080 | 960 GB/s | ~33.9 |
| RTX 5070 Ti | 896 GB/s | ~31.7 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Build it, or rent the compute?
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Frequently asked questions
Does Qwen3 32B fit in 24 GB?
Yes — Q4_K_M weights are 19.8 GB, leaving solid headroom for context on a single 24 GB card. Q5_K_M at 23.2 GB is tighter but workable at moderate context lengths.
What does "thinking mode" cost in VRAM or speed?
Thinking mode does not change the weights or VRAM footprint — it changes how many tokens the model generates before its final answer, so it costs generation time and context budget, not extra memory.
Qwen3 32B or DeepSeek-R1-Distill-Qwen-32B — which one?
Both are 32B-class and fit the same VRAM tiers. The DeepSeek distill leans harder into always-on chain-of-thought reasoning; Qwen3 32B gives you a fast non-thinking mode for everyday chat and a slower thinking mode on demand — pick based on whether you want one mode or both.