gpt-oss-120b
OpenAI's first open-weight model since GPT-2 — a 117B mixture-of-experts with only 5.1B active parameters, shipped already compressed to 4 bits, which makes its "quant options" behave unlike anything else in this registry.
Last verified 2026-07 · Page created; quant sizes verified against unsloth/gpt-oss-120b-GGUF — the near-identical sizes across quants reflect OpenAI's native MXFP4 release, not a data error.
What it is good at, and why the quants look strange
gpt-oss-120b was OpenAI's August 2025 return to open weights: a 128-expert mixture-of-experts model with 117B total parameters but only 5.1B active per token, released under a plain Apache 2.0 license. OpenAI shipped it natively quantized — the MoE expert weights are already packed at roughly 4 bits (MXFP4) at the source, which is why re-quantizing it with llama.cpp barely moves the needle: Q4_K_M, Q5_K_M, Q8_0, and even the "F16" upcast all land within a few GB of each other (62.8-65.4 GB), because the bulk of the file is already 4-bit and quantizing on top of that has little left to compress.
Practically: there is no meaningful "run it smaller" lever here the way there is for a normally-quantized model. The file size you get is close to the file size you are stuck with.
The hardware it wants
At ~63 GB regardless of quant choice, this does not fit a single 48 GB dual-3090 pool with room to spare, and OpenAI's own guidance is that it fits in a single 80 GB datacenter GPU. On consumer/prosumer hardware, the realistic paths are a Mac Studio or similar with 64-96 GB+ unified memory, or a 96-128 GB Strix Halo-class mini-PC — community guidance puts the practical floor around 64-66 GB of combined VRAM+RAM for reasonable (if not fast) throughput.
VRAM by quantization
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 65.4 GB | 92.0 GB | 142.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q8_0 | 63.4 GB | 89.7 GB | 140.2 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q5_K_M | 62.9 GB | 89.2 GB | 139.6 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q4_K_M | 62.8 GB | 89.0 GB | 139.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
Runs on
No catalog GPU in our reference list fits this model yet — check the full AI-scored catalog.
Recommended builds
Three ways to size a machine for gpt-oss-120b, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
Tokens/sec expectations
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~423.8 |
| RTX 4090 | 1008 GB/s | ~238.4 |
| RTX 6000 Ada | 960 GB/s | ~227 |
| RX 7900 XTX | 960 GB/s | ~227 |
| RTX 5080 | 960 GB/s | ~227 |
| RTX 5070 Ti | 896 GB/s | ~211.9 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Build it, or rent the compute?
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Frequently asked questions
Why don't the different quants save much space for this model?
OpenAI released the weights already natively quantized to roughly 4 bits (MXFP4) for the MoE expert layers. Re-quantizing an already-4-bit file with llama.cpp has little left to compress, so Q4_K_M through the F16 upcast all land in a narrow 62.8-65.4 GB band instead of the usual 3-4x spread.
Can I run gpt-oss-120b on two 24 GB GPUs?
Not comfortably — the file is roughly 63 GB regardless of quant, so 48 GB pooled falls short. You need closer to 64 GB+ of combined VRAM/unified memory, which points toward a single 80 GB GPU or a larger unified-memory machine rather than a two-card 24 GB rig.
How does the 5.1B active parameter count affect speed?
Generation speed tracks the 5.1B active parameters, not the 117B total — so once the model is loaded, throughput behaves much closer to a small dense model than a 117B one, even though you needed 117B worth of memory to load it.