OpenAI’s first open-weight model since GPT-2 — a 117B mixture-of-experts with only 5.1B active parameters, shipped already compressed to 4 bits, which makes its "quant options" behave unlike anything else in this registry.
Last verified 2026-07 · Page created; quant sizes verified against unsloth/gpt-oss-120b-GGUF — the near-identical sizes across quants reflect OpenAI’s native MXFP4 release, not a data error.
gpt-oss-120b was OpenAI’s August 2025 return to open weights: a 128-expert mixture-of-experts model with 117B total parameters but only 5.1B active per token, released under a plain Apache 2.0 license. OpenAI shipped it natively quantized — the MoE expert weights are already packed at roughly 4 bits (MXFP4) at the source, which is why re-quantizing it with llama.cpp barely moves the needle: Q4_K_M, Q5_K_M, Q8_0, and even the "F16" upcast all land within a few GB of each other (62.8–65.4GB), because the bulk of the file is already 4-bit and quantizing on top of that has little left to compress.
Practically: there is no meaningful "run it smaller" lever here the way there is for a normally-quantized model. The file size you get is close to the file size you are stuck with.
At ~63GB regardless of quant choice, this does not fit a single 48GB dual-3090 pool with room to spare, and OpenAI’s own guidance is that it fits in a single 80GB datacenter GPU. On consumer/prosumer hardware, the realistic paths are a Mac Studio or similar with 64–96GB+ unified memory, or a 96–128GB Strix Halo-class mini-PC — community guidance puts the practical floor around 64–66GB of combined VRAM+RAM for reasonable (if not fast) throughput.
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 65.4 GB | 92.0 GB | 142.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q8_0 | 63.4 GB | 89.7 GB | 140.2 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q5_K_M | 62.9 GB | 89.2 GB | 139.6 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q4_K_M | 62.8 GB | 89.0 GB | 139.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
No catalog GPU in our reference list fits this model yet — check the full AI-scored catalog.
Three ways to size a machine for gpt-oss-120b, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~423.8 |
| RTX 4090 | 1008 GB/s | ~238.4 |
| RTX 6000 Ada | 960 GB/s | ~227 |
| RX 7900 XTX | 960 GB/s | ~227 |
| RTX 5080 | 960 GB/s | ~227 |
| RTX 5070 Ti | 896 GB/s | ~211.9 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
OpenAI released the weights already natively quantized to roughly 4 bits (MXFP4) for the MoE expert layers. Re-quantizing an already-4-bit file with llama.cpp has little left to compress, so Q4_K_M through the F16 upcast all land in a narrow 62.8–65.4GB band instead of the usual 3–4x spread.
Not comfortably — the file is roughly 63GB regardless of quant, so 48GB pooled falls short. You need closer to 64GB+ of combined VRAM/unified memory, which points toward a single 80GB GPU or a larger unified-memory machine rather than a two-card 24GB rig.
Generation speed tracks the 5.1B active parameters, not the 117B total — so once the model is loaded, throughput behaves much closer to a small dense model than a 117B one, even though you needed 117B worth of memory to load it.