Pick the Gemma 4 size that fits your VRAM
getting started
// what it does
Five sizes span edge to server: E2B (2.3B effective) and E4B (4.5B effective) for phones and laptops with a 128K context; 12B unified multimodal; 26B A4B, a Mixture-of-Experts that activates only 3.8B parameters per token for near-31B quality at ~4B speed; and 31B dense, the strongest — the last three carry a 256K context. Rough guide: under 5 GB pick E2B (4-bit), 5-8 GB E4B (4-bit), ~8 GB 12B (4-bit), 24-32 GB 31B (4-bit), 48-80 GB 31B (bf16). Choose 26B A4B when you want high throughput on a server.
// gotcha
26B A4B is a Mixture-of-Experts, not a dense 26B: it is faster than its size suggests but still loads all 25.2B parameters into memory, so plan RAM/VRAM around the full weight, not the 3.8B active per token.