Phi-4
Microsoft's proof that training-data quality beats raw parameter count: a 14.7B model that beat much larger ones on math and reasoning benchmarks, MIT-licensed, and light enough for a 12 GB GPU.
Last verified 2026-07 · Page created; Q4_K_M/Q5_K_M/Q8_0/FP16 sizes verified against bartowski/phi-4-GGUF.
What it is good at
Phi-4 is Microsoft Research's December 2024 release built around a "quality of training data over scale" thesis — heavy use of synthetic and curated data got a 14.7B dense model to punch above models several times its size on math and STEM reasoning specifically. It was released under MIT, no usage restrictions.
The tradeoff for that focus is a modest 16K native context window — fine for chat and single-document tasks, a real constraint if you need to feed it long documents or large codebases at once.
The hardware it wants
Q4_K_M is 9.05 GB, comfortably fitting an 8 GB card with modest context or a 12 GB card with real headroom. Q5_K_M (10.6 GB) and Q8_0 (15.58 GB) both fit a 16 GB GPU with room to spare. This is one of the lighter models in this registry — a genuinely budget-friendly local LLM for math/code work, not just a scaled-down chat model.
VRAM by quantization
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 29.3 GB | 35.8 GB | 42.2 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Fits comfortably |
| Q8_0 | 15.6 GB | 20.0 GB | 26.4 GB | Doesn't fit | Doesn't fit | Fits comfortably | Fits comfortably | Fits comfortably |
| Q5_K_M | 10.6 GB | 14.3 GB | 20.6 GB | Doesn't fit | Fits comfortably | Fits comfortably | Fits comfortably | Fits comfortably |
| Q4_K_M | 9.1 GB | 12.5 GB | 18.9 GB | Doesn't fit | Fits comfortably | Fits comfortably | Fits comfortably | Fits comfortably |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
Runs on
Unified-memory machines — Apple Silicon Mac Studio/MacBook Pro (Max/Ultra chips) and AMD Strix Halo mini-PCs — share system RAM and GPU memory in one pool, so a 64–128 GB configuration can hold models that would otherwise need multiple discrete GPUs. Throughput is bandwidth-bound like any GPU; it isn’t modeled in the table above yet.
Recommended builds
Three ways to size a machine for Phi-4, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
Tokens/sec expectations
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~138.6 |
| RTX 4090 | 1008 GB/s | ~78 |
| RTX 6000 Ada | 960 GB/s | ~74.3 |
| RX 7900 XTX | 960 GB/s | ~74.3 |
| RTX 5080 | 960 GB/s | ~74.3 |
| RTX 5070 Ti | 896 GB/s | ~69.3 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Build it, or rent the compute?
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Frequently asked questions
What GPU do I need for Phi-4?
An 8 GB card handles Q4_K_M with modest context; a 12-16 GB card runs any quant up to Q8_0 comfortably. This is one of the more accessible models here for a budget AI build.
Is Phi-4 good for coding, or just math?
Math and STEM reasoning are its strongest suit by design, but it holds up reasonably for general coding tasks too — it is not a specialized code-only model, just one where the training data leaned hard into structured/reasoning-heavy content.
Is the 16K context window a real limitation?
For chat and single-file coding tasks, rarely. For feeding it a long document or a multi-file codebase in one go, yes — 16K fills up faster than the 32K-128K windows on most other models in this registry.