Llama 3.3 70B hardware requirements
About the model
- Parameters
- 70.6B
- Attention
- Full attention
- Max context
- 131,072 tokens
- Released
- 2024-12-06
- Model card
- Llama 3.3 70B on Hugging Face
GGUF files
| Quant | File size | Quality | Published by |
|---|---|---|---|
| IQ2_M | 24.12 GB | Heavy loss | Community GGUF · bartowski |
| Q2_K | 26.38 GB | Heavy loss | Community GGUF · bartowski |
| IQ4_XS | 37.90 GB | Good balance | Community GGUF · bartowski |
| Q4_K_Mbaseline | 42.52 GB | Good balance | Community GGUF · bartowski |
| Q8_0 | 74.98 GB | Near-lossless | Community GGUF · bartowski |
Memory needed by quant and context
| Quant | 4k | 8k | 16k | 32k | 64k | 128k |
|---|---|---|---|---|---|---|
| IQ2_M | 26.0 | 27.4 | 30.1 | 35.7 | 46.7 | 68.8 |
| Q2_K | 28.3 | 29.6 | 32.4 | 37.9 | 49.0 | 71.0 |
| IQ4_XS | 39.8 | 41.2 | 43.9 | 49.4 | 60.5 | 82.5 |
| Q4_K_M | 44.4 | 45.8 | 48.5 | 54.1 | 65.1 | 87.2 |
| Q8_0 | 76.9 | 78.2 | 81.0 | 86.5 | 97.6 | 119.6 |
Which GPUs can run Llama 3.3 70B?
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Dense models split between GPU and CPU slow down sharply — the CPU side sets the pace. MoE models that keep only their experts in system RAM degrade far more gently.
Scroll sideways to see every column.
| Hardware | Verdict | Speed | Quant | Memory | Runs as | Context | Notes | Price |
|---|---|---|---|---|---|---|---|---|
| Apple M5 Max (40-core GPU) · 128 GB | Runs well | est. 8.1 tok/s6.5–9.8calibrated estimate ±20% | Q4_K_M | 45.8 / 89.6 GB | Unified memory | up to 8k | — | — |
| Apple M3 Ultra · 96 GB | Runs slowly | est. 8.0 tok/s6.4–9.6calibrated estimate ±20% | Q4_K_M | 45.8 / 67.2 GB | Unified memory | up to 64k |
| — |
| Apple M2 Ultra · 64 GB | Runs slowly | est. 6.0 tok/s4.2–7.8theoretical estimate ±30% | Q4_K_M | 45.2 GB RAM | CPU only | up to 32k |
| — |
| Apple M4 Max (40-core GPU) · 64 GB | Runs slowly | est. 4.2 tok/s2.9–5.5theoretical estimate ±30% | Q4_K_M | 45.2 GB RAM | CPU only | up to 32k |
| — |
| Ryzen AI Max+ 395 (Strix Halo) · 64 GB | Runs slowly | est. 2.8 tok/s2.0–3.7theoretical estimate ±30% | Q4_K_M | 45.2 GB RAM | CPU only | up to 32k | — | $1,999 launch MSRP |
| GeForce RTX 5090 | Runs slowly | est. 2.7 tok/s2.2–3.3calibrated estimate ±20% | Q4_K_M | 32.0 / 32.0 GB + 14.4 GB RAM | Partial offload | up to 16k |
| $4,200 street (as of 2026-08-09) |
| NVIDIA DGX Spark · 128 GB | Runs slowly | est. 2.4 tok/s1.9–2.9calibrated estimate ±20% | Q4_K_M | 45.8 / 89.6 GB | Unified memory | up to 32k | — | $3,999 launch MSRP |
| Apple M5 Pro · 64 GB | Runs slowly | est. 2.4 tok/s1.7–3.1theoretical estimate ±30% | Q4_K_M | 45.2 GB RAM | CPU only | up to 32k | — | — |
| Apple M4 Pro · 64 GB | Runs slowly | est. 2.1 tok/s1.5–2.7theoretical estimate ±30% | Q4_K_M | 45.2 GB RAM | CPU only | up to 8k | — | — |
| Apple M5 Max (32-core GPU) · 48 GB | Won't runTry IQ4_XS: Runs slowly | est. 4.0 tok/s2.8–5.1theoretical estimate ±30% | IQ4_XS | 40.6 GB RAM | CPU only | up to 16k |
| — |
| Radeon RX 7900 XT | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 3.6 tok/s2.9–4.3calibrated estimate ±20% | Q2_K | 20.0 / 20.0 GB + 10.2 GB RAM | Partial offload | up to 16k | — | $899 launch MSRP |
| Apple M4 Max (32-core GPU) · 48 GB | Won't runTry IQ4_XS: Runs slowly | est. 3.5 tok/s2.5–4.6theoretical estimate ±30% | IQ4_XS | 40.6 GB RAM | CPU only | up to 16k |
| — |
| GeForce RTX 5080 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.2–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $1,256 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.2–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $949 street (as of 2026-08-09) |
| GeForce RTX 5080 Laptop | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.2–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | — |
| GeForce RTX 4080 Super | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $999 launch MSRP |
| Radeon RX 9070 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $639 street (as of 2026-08-09) |
| Radeon RX 9070 XT | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $689 street (as of 2026-08-09) |
| GeForce RTX 4080 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $1,199 launch MSRP |
| Radeon RX 7800 XT | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.6 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $499 launch MSRP |
| GeForce RTX 4070 Ti Super | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.6 tok/s2.1–3.2calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $799 launch MSRP |
| GeForce RTX 4090 Laptop | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.6 tok/s2.1–3.1calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | — |
| GeForce RTX 5060 Ti 16GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.5 tok/s2.0–3.1calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 16k | — | $569 street (as of 2026-08-09) |
| Radeon RX 9060 XT 16GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.5 tok/s2.0–3.0calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 8k | — | $449 street (as of 2026-08-09) |
| GeForce RTX 4060 Ti 16GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.4 tok/s1.9–2.9calibrated estimate ±20% | Q2_K | 16.0 / 16.0 GB + 14.2 GB RAM | Partial offload | up to 8k | — | $499 launch MSRP |
| Radeon RX 7900 XTX | Won't runTry IQ4_XS: Runs slowly | est. 2.2 tok/s1.8–2.6calibrated estimate ±20% | IQ4_XS | 24.0 / 24.0 GB + 17.8 GB RAM | Partial offload | up to 8k | — | $999 launch MSRP |
| GeForce RTX 3090 Ti | Won't runTry IQ4_XS: Runs slowly | est. 2.2 tok/s1.8–2.6calibrated estimate ±20% | IQ4_XS | 24.0 / 24.0 GB + 17.8 GB RAM | Partial offload | up to 8k | — | $1,999 launch MSRP |
| GeForce RTX 4090 | Won't runTry IQ4_XS: Runs slowly | est. 2.2 tok/s1.8–2.6calibrated estimate ±20% | IQ4_XS | 24.0 / 24.0 GB + 17.8 GB RAM | Partial offload | up to 8k | — | $2,755 street (as of 2026-08-09) |
| GeForce RTX 3090 | Won't runTry IQ4_XS: Runs slowly | est. 2.2 tok/s1.7–2.6calibrated estimate ±20% | IQ4_XS | 24.0 / 24.0 GB + 17.8 GB RAM | Partial offload | up to 8k | — | $1,050 street (as of 2026-08-09) |
| GeForce RTX 3080 10GB | Won't runTry IQ2_M (heavy quality loss): Runs slowly | est. 2.2 tok/s1.7–2.6calibrated estimate ±20% | IQ2_M | 10.0 / 10.0 GB + 18.0 GB RAM | Partial offload | up to 8k | — | $699 launch MSRP |
| GeForce RTX 5090 Laptop | Won't runTry IQ4_XS: Runs slowly | est. 2.2 tok/s1.7–2.6calibrated estimate ±20% | IQ4_XS | 24.0 / 24.0 GB + 17.8 GB RAM | Partial offload | up to 8k | — | — |
| GeForce RTX 3080 12GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.2 tok/s1.7–2.6calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $799 launch MSRP |
| GeForce RTX 5070 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.6calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $629 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti Laptop | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.6calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | — |
| GeForce RTX 4070 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.5calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $599 launch MSRP |
| GeForce RTX 4070 Super | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.5calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $599 launch MSRP |
| GeForce RTX 4080 Laptop | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.5calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | — |
| Intel Arc B580 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.5calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $290 street (as of 2026-08-09) |
| GeForce RTX 3060 12GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.1 tok/s1.7–2.5calibrated estimate ±20% | Q2_K | 12.0 / 12.0 GB + 18.2 GB RAM | Partial offload | up to 8k | — | $250 street (as of 2026-08-09) |
| GeForce RTX 2080 Ti | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.0 tok/s1.6–2.4calibrated estimate ±20% | Q2_K | 11.0 / 11.0 GB + 19.2 GB RAM | Partial offload | up to 8k | — | $999 launch MSRP |
| GeForce GTX 1080 Ti | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.0 tok/s1.4–2.6theoretical estimate ±30% | Q2_K | 11.0 / 11.0 GB + 19.2 GB RAM | Partial offload | up to 8k | — | $699 launch MSRP |
| DDR5-6000 dual-channel CPU · 96 GB | Won't run | est. 1.1 tok/s0.8–1.3calibrated estimate ±20% | Q4_K_M | 45.2 GB RAM | CPU only | — |
| — |
| DDR5-5600 dual-channel CPU · 96 GB | Won't run | est. 1.0 tok/s0.8–1.2calibrated estimate ±20% | Q4_K_M | 45.2 GB RAM | CPU only | — |
| — |
| DDR4-3200 dual-channel CPU · 64 GB | Won't run | est. 0.6 tok/s0.5–0.7calibrated estimate ±20% | Q4_K_M | 45.2 GB RAM | CPU only | — |
| — |
| Apple M4 · 32 GB | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| — |
| Apple M5 · 32 GB | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| — |
| GeForce RTX 3050 8GB | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $249 launch MSRP |
| GeForce RTX 3070 | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $499 launch MSRP |
| GeForce RTX 4060 | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $299 launch MSRP |
| GeForce RTX 4060 Laptop | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| — |
| GeForce RTX 4060 Ti 8GB | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $399 launch MSRP |
| GeForce RTX 4070 Laptop | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| — |
| GeForce RTX 5060 | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $339 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 8GB | Won't run | — | Q4_K_M | needs 45.8 GB | — | — |
| $429 street (as of 2026-08-09) |
Cheapest GPUs that run it
No GPU with a current street price reaches Runs great or Runs well at Q4_K_M. The table above lists Macs, unified-memory PCs and CPU setups.
Measured results for Llama 3.3 70B
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Hardware | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| Apple M3 Max 128GB | Q4_K_M | ollama | 2k | — | 9.8 | — | heyuan110.com | 2026-04-14 |
Frequently asked questions
How much VRAM does Llama 3.3 70B need?
At Q4_K_M the weights are 42.52 GB; with 8k context the total is about 45.8 GB (KV cache 2.7 GB, compute buffer 0.6 GB). The smallest tracked file (IQ2_M (heavy quality loss)) needs about 27.4 GB.
What is the cheapest GPU that runs Llama 3.3 70B well?
No GPU with a current street price reaches Runs great or Runs well at Q4_K_M. The best tracked option is Apple M5 Max (40-core GPU) · 128 GB: Runs well.
Can Llama 3.3 70B run on an 8 GB, 16 GB or 24 GB GPU?
At Q4_K_M with 8k context: GeForce RTX 4060 — Won't run; GeForce RTX 5060 Ti 16GB — Won't run; GeForce RTX 3090 — Won't run, est. 1.8 tok/s (1.4–2.1, calibrated estimate ±20%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.