> fine-tuning & quantization
Gemma 4 fine-tunes cheaply with QLoRA: load the base in 4-bit, train small LoRA adapters, and an E4B fits a single 16 GB GPU. The trade-offs are model-specific — the 26B MoE needs LoRA (not full fine-tuning) to protect expert routing, and vision layers should stay frozen for text-only runs. For inference, Google's QAT int4 checkpoints give near-bf16 quality at a third of the memory.
// fine-tuning & quantization
4 commands// faq
Can I fine-tune Gemma 4 on a free Colab GPU?
Yes, with QLoRA. Loading E4B in 4-bit and training LoRA adapters (r=16, lora_alpha=16 on the attention and MLP projections) fits a single 16 GB T4 on free Colab or Kaggle. Unsloth's FastModel wraps this with Gemma 4 defaults. Keep chain-of-thought examples the majority of your data or thinking mode degrades.
How is fine-tuning the 26B A4B different?
It is a Mixture-of-Experts that routes each token to 8 of 128 experts. Use LoRA in bf16 and do not target the router weights — a full fine-tune or router adapters can collapse the routing so a few experts dominate, crippling the model. Increase context length gradually after the loss settles.
What are QAT checkpoints and when should I use them?
Quantization-Aware Trained int4 checkpoints (google/gemma-4-<size>-it-qat-q4_0-gguf) are trained to tolerate 4-bit weights, so they retain most bf16 quality at roughly a third of the memory. Use them for inference when you want the smallest footprint without the quality drop of naive post-training quantization — but never re-quantize them on top.
Should I train the vision encoder when fine-tuning on text?
No. Freeze the vision tower (finetune_vision_layers=False in Unsloth) and train only the language and projection layers. Updating the encoder on text-only data silently erodes image understanding while the loss still looks fine. Unfreeze it only to adapt to a specific visual domain, and validate text quality first.