Gemma 4 12B hardware requirements
About the model
- Parameters
- 11.96B
- Attention
- Sliding-window attention
- Max context
- 262,144 tokens
- Released
- 2026-04-01
- Model card
- Gemma 4 12B on Hugging Face
GGUF files
| Quant | File size | Quality | Published by |
|---|---|---|---|
| UD-Q2_K_XL | 4.66 GB | Heavy loss | Community GGUF · unsloth |
| IQ4_XS | 6.38 GB | Good balance | Community GGUF · unsloth |
| Q4_K_Mbaseline | 7.12 GB | Good balance | Community GGUF · unsloth |
| Q8_0 | 12.67 GB | Near-lossless | Community GGUF · unsloth |
File sizes do not include the vision projector (mmproj) that image input needs.
Some repositories publish only Unsloth Dynamic (UD) variants; sizes are those of the file you would download.
Memory needed by quant and context
| Quant | 4k | 8k | 16k | 32k | 64k | 128k | 256k |
|---|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 5.5 | 5.8 | 6.4 | 7.6 | 10.1 | 14.9 | 24.7 |
| IQ4_XS | 7.2 | 7.5 | 8.1 | 9.3 | 11.8 | 16.7 | 26.5 |
| Q4_K_M | 7.9 | 8.2 | 8.8 | 10.1 | 12.5 | 17.4 | 27.2 |
| Q8_0 | 13.5 | 13.8 | 14.4 | 15.6 | 18.1 | 23.0 | 32.7 |
Which GPUs can run Gemma 4 12B?
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Dense models split between GPU and CPU slow down sharply — the CPU side sets the pace. MoE models that keep only their experts in system RAM degrade far more gently.
Scroll sideways to see every column.
| Hardware | Verdict | Speed | Quant | Memory | Runs as | Context | Notes | Price |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 5090 | Runs great | est. 163.8 tok/s144.2–183.5calibrated estimate ±12% | Q4_K_M | 8.8 / 32.0 GB | Full GPU | up to 256k | — | $4,200 street (as of 2026-08-09) |
| Radeon RX 7900 XTX | Runs great | est. 97.8 tok/s86.1–109.5calibrated estimate ±12% | Q4_K_M | 8.8 / 24.0 GB | Full GPU | up to 128k | — | $999 launch MSRP |
| GeForce RTX 3090 Ti | Runs great | est. 92.2 tok/s81.1–103.2calibrated estimate ±12% | Q4_K_M | 8.8 / 24.0 GB | Full GPU | up to 128k | — | $1,999 launch MSRP |
| GeForce RTX 4090 | Runs great | est. 92.2 tok/s81.1–103.2calibrated estimate ±12% | Q4_K_M | 8.8 / 24.0 GB | Full GPU | up to 128k | — | $2,755 street (as of 2026-08-09) |
| GeForce RTX 5080 | Runs great | est. 87.8 tok/s77.2–98.3calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $1,256 street (as of 2026-08-09) |
| GeForce RTX 3090 | Runs great | est. 85.6 tok/s75.3–95.8calibrated estimate ±12% | Q4_K_M | 8.8 / 24.0 GB | Full GPU | up to 128k | — | $1,050 street (as of 2026-08-09) |
| GeForce RTX 3080 12GB | Runs great | est. 83.4 tok/s73.4–93.4calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $799 launch MSRP |
| GeForce RTX 5070 Ti | Runs great | est. 81.9 tok/s72.1–91.7calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $949 street (as of 2026-08-09) |
| GeForce RTX 5080 Laptop | Runs great | est. 81.9 tok/s72.1–91.7calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | — |
| GeForce RTX 5090 Laptop | Runs great | est. 81.9 tok/s72.1–91.7calibrated estimate ±12% | Q4_K_M | 8.8 / 24.0 GB | Full GPU | up to 128k | — | — |
| Radeon RX 7900 XT | Runs great | est. 81.5 tok/s71.7–91.3calibrated estimate ±12% | Q4_K_M | 8.8 / 20.0 GB | Full GPU | up to 128k | — | $899 launch MSRP |
| GeForce RTX 3080 10GB | Runs great | est. 69.5 tok/s61.1–77.8calibrated estimate ±12% | Q4_K_M | 8.8 / 10.0 GB | Full GPU | up to 16k | — | $699 launch MSRP |
| GeForce RTX 4080 Super | Runs great | est. 67.3 tok/s59.2–75.4calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $999 launch MSRP |
| Radeon RX 9070 | Runs great | est. 65.7 tok/s57.8–73.6calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $639 street (as of 2026-08-09) |
| Radeon RX 9070 XT | Runs great | est. 65.7 tok/s57.8–73.6calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $689 street (as of 2026-08-09) |
| GeForce RTX 4080 | Runs great | est. 65.5 tok/s57.7–73.4calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $1,199 launch MSRP |
| Radeon RX 7800 XT | Runs great | est. 63.6 tok/s55.9–71.2calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $499 launch MSRP |
| GeForce RTX 4070 Ti Super | Runs great | est. 61.4 tok/s54.1–68.8calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $799 launch MSRP |
| GeForce RTX 5070 | Runs great | est. 61.4 tok/s54.1–68.8calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $629 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti Laptop | Runs great | est. 61.4 tok/s54.1–68.8calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | — |
| GeForce RTX 2080 Ti | Runs great | est. 56.3 tok/s49.6–63.1calibrated estimate ±12% | Q4_K_M | 8.8 / 11.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| GeForce RTX 4090 Laptop | Runs great | est. 52.7 tok/s46.3–59.0calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | — |
| Apple M5 Max (40-core GPU) · 48 GB | Runs great | est. 48.1 tok/s38.5–57.7calibrated estimate ±20% | Q4_K_M | 8.2 / 33.6 GB | Unified memory | up to 128k | — | — |
| Apple M3 Ultra · 96 GB | Runs great | est. 47.1 tok/s37.7–56.5calibrated estimate ±20% | Q4_K_M | 8.2 / 67.2 GB | Unified memory | up to 128k | — | — |
| GeForce RTX 4070 | Runs great | est. 46.1 tok/s40.5–51.6calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $599 launch MSRP |
| GeForce RTX 4070 Super | Runs great | est. 46.1 tok/s40.5–51.6calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $599 launch MSRP |
| Apple M2 Ultra · 64 GB | Runs great | est. 46.0 tok/s36.8–55.2calibrated estimate ±20% | Q4_K_M | 8.2 / 44.8 GB | Unified memory | up to 128k | — | — |
| GeForce GTX 1080 Ti | Runs great | est. 44.2 tok/s31.0–57.5theoretical estimate ±30% | Q4_K_M | 8.8 / 11.0 GB | Full GPU | up to 32k | — | $699 launch MSRP |
| Apple M4 Max (40-core GPU) · 48 GB | Runs great | est. 42.8 tok/s34.2–51.3calibrated estimate ±20% | Q4_K_M | 8.2 / 33.6 GB | Unified memory | up to 128k | — | — |
| GeForce RTX 5060 Ti 16GB | Runs great | est. 41.0 tok/s36.0–45.9calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $569 street (as of 2026-08-09) |
| GeForce RTX 4080 Laptop | Runs great | est. 39.5 tok/s34.8–44.2calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | — |
| Apple M5 Max (32-core GPU) · 36 GB | Runs great | est. 36.0 tok/s28.8–43.3calibrated estimate ±20% | Q4_K_M | 8.2 / 25.2 GB | Unified memory | up to 64k | — | — |
| Intel Arc B580 | Runs great | est. 35.1 tok/s28.1–42.2calibrated estimate ±20% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $290 street (as of 2026-08-09) |
| GeForce RTX 3060 12GB | Runs great | est. 32.9 tok/s29.0–36.9calibrated estimate ±12% | Q4_K_M | 8.8 / 12.0 GB | Full GPU | up to 32k | — | $250 street (as of 2026-08-09) |
| Radeon RX 9060 XT 16GB | Runs great | est. 32.8 tok/s28.9–36.7calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 64k | — | $449 street (as of 2026-08-09) |
| Apple M4 Max (32-core GPU) · 36 GB | Runs great | est. 32.1 tok/s25.7–38.6calibrated estimate ±20% | Q4_K_M | 8.2 / 25.2 GB | Unified memory | up to 64k | — | — |
| GeForce RTX 4060 Ti 16GB | Runs great | est. 26.3 tok/s23.2–29.5calibrated estimate ±12% | Q4_K_M | 8.8 / 16.0 GB | Full GPU | up to 32k | — | $499 launch MSRP |
| Apple M5 Pro · 24 GB | Runs great | est. 24.1 tok/s19.2–28.9calibrated estimate ±20% | Q4_K_M | 8.2 / 16.8 GB | Unified memory | up to 16k | — | — |
| Ryzen AI Max+ 395 (Strix Halo) · 32 GB | Runs great | est. 21.7 tok/s17.4–26.1calibrated estimate ±20% | Q4_K_M | 8.2 / 22.4 GB | Unified memory | up to 16k | — | $1,999 launch MSRP |
| Apple M4 Pro · 24 GB | Runs great | est. 21.4 tok/s17.1–25.7calibrated estimate ±20% | Q4_K_M | 8.2 / 16.8 GB | Unified memory | up to 8k | — | — |
| NVIDIA DGX Spark · 128 GB | Runs well | est. 14.3 tok/s11.4–17.1calibrated estimate ±20% | Q4_K_M | 8.2 / 89.6 GB | Unified memory | up to 64k |
| $3,999 launch MSRP |
| Apple M5 · 16 GB | Runs well | est. 12.0 tok/s9.6–14.4calibrated estimate ±20% | Q4_K_M | 8.2 / 11.2 GB | Unified memory | up to 32k | — | — |
| Apple M4 · 16 GB | Runs well | est. 9.4 tok/s7.5–11.3calibrated estimate ±20% | Q4_K_M | 8.2 / 11.2 GB | Unified memory | up to 16k | — | — |
| GeForce RTX 3070 | Runs slowly | est. 24.1 tok/s19.3–28.9calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $499 launch MSRP |
| GeForce RTX 5060 | Runs slowly | est. 24.1 tok/s19.3–28.9calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $339 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 8GB | Runs slowly | est. 24.1 tok/s19.3–28.9calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $429 street (as of 2026-08-09) |
| GeForce RTX 4060 Ti 8GB | Runs slowly | est. 18.7 tok/s15.0–22.4calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $399 launch MSRP |
| GeForce RTX 4060 | Runs slowly | est. 18.0 tok/s14.4–21.6calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $299 launch MSRP |
| GeForce RTX 4060 Laptop | Runs slowly | est. 17.3 tok/s13.9–20.8calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| — |
| GeForce RTX 4070 Laptop | Runs slowly | est. 17.3 tok/s13.9–20.8calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| — |
| GeForce RTX 3050 8GB | Runs slowly | est. 15.8 tok/s12.7–19.0calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 0.8 GB RAM | Partial offload | up to 64k |
| $249 launch MSRP |
| DDR5-6000 dual-channel CPU · 16 GB | Runs slowly | est. 6.3 tok/s5.0–7.5calibrated estimate ±20% | Q4_K_M | 7.7 GB RAM | CPU only | up to 64k | — | — |
| DDR5-5600 dual-channel CPU · 16 GB | Runs slowly | est. 5.9 tok/s4.7–7.0calibrated estimate ±20% | Q4_K_M | 7.7 GB RAM | CPU only | up to 64k | — | — |
| DDR4-3200 dual-channel CPU · 16 GB | Runs slowly | est. 3.3 tok/s2.7–4.0calibrated estimate ±20% | Q4_K_M | 7.7 GB RAM | CPU only | up to 64k | — | — |
Cheapest GPUs that run it
- Runs greatGeForce RTX 3060 12GB$250 street (as of 2026-08-09), est. 32.9 tok/s (29.0–36.9, calibrated estimate ±12%).
Measured results for Gemma 4 12B
No public measurements for this model yet.
Frequently asked questions
How much VRAM does Gemma 4 12B need?
At Q4_K_M the weights are 7.12 GB; with 8k context the total is about 8.2 GB (KV cache 0.5 GB, compute buffer 0.6 GB). The smallest tracked file (UD-Q2_K_XL (heavy quality loss)) needs about 5.8 GB.
What is the cheapest GPU that runs Gemma 4 12B well?
Cheapest that runs it great: GeForce RTX 3060 12GB at $250 (street price as of 2026-08-09), est. 32.9 tok/s (29.0–36.9, calibrated estimate ±12%).
Can Gemma 4 12B run on an 8 GB, 16 GB or 24 GB GPU?
At Q4_K_M with 8k context: GeForce RTX 4060 — Runs slowly, est. 18.0 tok/s (14.4–21.6, calibrated estimate ±20%); GeForce RTX 5060 Ti 16GB — Runs great, est. 41.0 tok/s (36.0–45.9, calibrated estimate ±12%); GeForce RTX 3090 — Runs great, est. 85.6 tok/s (75.3–95.8, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.