DeepSeek-R1-Distill-Qwen-32B
The practical way most people actually run "DeepSeek-R1" at home: a 32B dense model distilled from R1's reasoning traces onto a Qwen2.5 base, sized to fit a single 24 GB GPU.
Last verified 2026-07 · Page created; Q4_K_M/Q5_K_M/Q8_0/FP16(BF16) sizes verified against bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF.
What it is good at
This is a Qwen2.5-32B base fine-tuned on reasoning traces generated by full DeepSeek-R1, which is what lets a 32B dense model punch well above its parameter count on math, code, and multi-step logic — it inherits R1's "think before answering" style without needing R1's 671B footprint. It is MIT-licensed with no restriction on commercial use.
Being dense rather than MoE, every one of its 32.5B parameters activates per token, so throughput and VRAM scale exactly the way they do for any other 32B-class model — no expert-routing surprises to account for.
The hardware it wants
Q4_K_M is 19.85 GB, which comfortably fits a single 24 GB GPU (RTX 3090/4090) with real headroom for context and a chat session. Q5_K_M (23.26 GB) is a tighter fit on 24 GB once you account for KV cache — workable at shorter context, tight at the full window. Q8_0 (34.82 GB) needs a 48 GB card or two pooled 24 GB cards.
Because reasoning models "think out loud" before answering, effective context use runs higher than a normal chat model at the same nominal context length — budget more headroom than you would for a straight-answer 32B when picking a quant.
VRAM by quantization
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 65.5 GB | 80.0 GB | 94.1 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q8_0 | 34.8 GB | 44.7 GB | 58.7 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Fits comfortably |
| Q5_K_M | 23.3 GB | 31.4 GB | 45.4 GB | Doesn't fit | Doesn't fit | Doesn't fit | Fits comfortably | Fits comfortably |
| Q4_K_M | 19.9 GB | 27.5 GB | 41.5 GB | Doesn't fit | Doesn't fit | Tight — reduced context | Fits comfortably | Fits comfortably |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
Runs on
Unified-memory machines — Apple Silicon Mac Studio/MacBook Pro (Max/Ultra chips) and AMD Strix Halo mini-PCs — share system RAM and GPU memory in one pool, so a 64–128 GB configuration can hold models that would otherwise need multiple discrete GPUs. Throughput is bandwidth-bound like any GPU; it isn’t modeled in the table above yet.
Recommended builds
Three ways to size a machine for DeepSeek-R1-Distill-Qwen-32B, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
Tokens/sec expectations
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~63.2 |
| RTX 4090 | 1008 GB/s | ~35.5 |
| RTX 6000 Ada | 960 GB/s | ~33.9 |
| RX 7900 XTX | 960 GB/s | ~33.9 |
| RTX 5080 | 960 GB/s | ~33.9 |
| RTX 5070 Ti | 896 GB/s | ~31.6 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Build it, or rent the compute?
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Frequently asked questions
Does this fit in 24 GB?
Yes at Q4_K_M (19.85 GB weights) with room to spare for context. Q5_K_M (23.26 GB) is workable on 24 GB but tight once the reasoning chain and KV cache are accounted for — Q4_K_M is the safer default on a single 24 GB card.
Is this actually DeepSeek-R1, just compressed?
No — it is a separate, much smaller Qwen2.5-32B model fine-tuned on R1's reasoning outputs, not a quantized version of the 671B model. It carries a lot of R1's reasoning behavior but is not identical to running full R1.
What GPU should I buy for this specifically?
A single 24 GB card (RTX 3090 used, or RTX 4090/5070 Ti new) covers Q4_K_M and Q5_K_M with room for a real chat session. If you want Q8_0 headroom too, size up to 32-48 GB.