Meta tuned a 70B dense model to close most of the gap with the 405B flagship, and it was the release that made "run a 70B at home" a realistic weekend project instead of a server-room one.
Last verified 2026-07 · Page created; Q4_K_M/Q5_K_M/Q8_0/FP16 sizes verified against bartowski/Llama-3.3-70B-Instruct-GGUF.
Llama 3.3 70B replaced 3.1 70B in December 2024 as Meta’s flagship dense chat model, retrained specifically to close the gap with the much larger 405B on reasoning, coding, and multilingual output. It is a dense (non-MoE) model, so there are no routing quirks to work around — every one of its 70.6B parameters runs on every token, which is exactly why it is also the heaviest model in this registry to host.
Tooling support is as broad as it gets: llama.cpp, Ollama, vLLM, and LM Studio all treat it as a first-class target on day one, which matters once you are choosing a quant and a serving stack rather than fighting compatibility.
Q4_K_M weighs in at 42.5GB, which already rules out any single consumer GPU with headroom to spare — a 24GB card needs the remainder offloaded to system RAM, and that offload is where throughput collapses. The realistic single-box path is two 24GB GPUs (2x RTX 3090 or 4090) pooling to 48GB, or a single 48GB workstation card (RTX A6000 / 6000 Ada).
Apple Silicon with 64GB or more of unified memory runs the same quant comfortably in one pool with no multi-card wiring, at the usual cost of slower generation than a discrete GPU pair — unified memory trades bandwidth for capacity, and at 70B the capacity is what you need first.
Q4_K_M is the practical floor for this model — a 70B model that starts losing coherence on top of a 4-bit quant defeats the point of running it locally over a smaller, faster model. Q5_K_M (49.95GB) and Q8_0 (74.98GB) both push you into 48GB+ territory regardless, so if two 24GB cards is your ceiling, Q4_K_M is the quant to standardize on.
| Quant | File size | VRAM @ 8K ctx | VRAM @ 32K ctx | 8 GB | 16 GB | 24 GB | 32 GB | 48 GB |
|---|---|---|---|---|---|---|---|---|
| FP16 | 141.1 GB | 172.4 GB | 202.9 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q8_0 | 75.0 GB | 96.4 GB | 126.8 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q5_K_M | 50.0 GB | 67.6 GB | 98.0 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
| Q4_K_M | 42.5 GB | 59.0 GB | 89.5 GB | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit | Doesn't fit |
Fit verdicts use the 8K context preset. Estimated from published weight sizes plus standard KV-cache math — see methodology.
No catalog GPU in our reference list fits this model yet — check the full AI-scored catalog.
Three ways to size a machine for Llama 3.3 70B, from the smallest GPU that fits to the no-compromise option with headroom for the next model up.
| GPU | Bandwidth | Est. tok/s (Q4_K_M) |
|---|---|---|
| RTX 5090 | 1792 GB/s | ~29.5 |
| RTX 4090 | 1008 GB/s | ~16.6 |
| RTX 6000 Ada | 960 GB/s | ~15.8 |
| RX 7900 XTX | 960 GB/s | ~15.8 |
| RTX 5080 | 960 GB/s | ~15.8 |
| RTX 5070 Ti | 896 GB/s | ~14.8 |
Estimated from memory bandwidth ÷ active weight size — a ceiling, not a measured figure. Real throughput depends on the inference engine, batch size, and context length.
Owning the hardware is a one-time cost that keeps paying off the more you actually use it — every run after the first is free, the data never leaves your machine, and the GPU still does everything else a PC does (gaming, editing, the rest of your workload) between AI sessions. Renting cloud GPU time trades that upfront cost for pay-as-you-go pricing with no idle hardware to maintain, which wins for occasional, bursty, or much-larger-than-consumer-VRAM jobs. For a model that fits comfortably in the VRAM tiers above and gets used regularly, owning the GPU tends to be the cheaper path over time — for anything larger than a single card can hold, renting is often the more practical starting point.
Not at a usable speed. Q4_K_M weights alone are 42.5GB, so a single 24GB card needs the rest offloaded to system RAM, and that offload is what tanks throughput. Two 24GB cards pooling to 48GB is the realistic single-machine path.
For reasoning-heavy and multi-step tasks, yes — it is a genuinely different quality tier. But it needs roughly 2x the VRAM of a 32B-class model at the same quant (42.5GB vs 19.8GB at Q4_K_M), so if your hardware tops out at one 16–24GB card, a model like Qwen3-32B or Gemma 3 27B is the better fit.
Dense models split cleanly across GPUs by layer (tensor-parallel or simple layer-split in llama.cpp/vLLM), so two 24GB cards pooling to 48GB is a well-trodden path for this model specifically — it does not need MoE-style expert routing to benefit from a second card.