Load Gemma 4 in 4-bit to fit a small GPU
python & transformers
// what it does
Install bitsandbytes, then pass a BitsAndBytesConfig(load_in_4bit=True) as quantization_config to from_pretrained to cut weight memory roughly 4x versus bf16 (E4B drops from ~15 GB to ~5 GB). It quantizes on the fly at load time. For pre-quantized weights that skip this step, use Google's QAT GGUFs instead.
// shell
$ pip install -U bitsandbytes// gotcha
bitsandbytes needs a CUDA GPU — it will not 4-bit-quantize on Apple Silicon or CPU. On a Mac, reach for the MLX or GGUF path instead.