> gemma 4
Google's open-weights model family, Apache 2.0 — run it with Ollama or llama.cpp, serve it as an OpenAI-compatible API, steer thinking mode, and fine-tune on your own data
// getting started
5 commands// run with ollama
6 commandsPull a specific Gemma 4 size with Ollama
$ ollama pull gemma4:31bChat with Gemma 4 interactively
$ ollama run gemma4Caption or read text from an image
$ ollama run gemma4 "describe ./chart.png"Run the Apple Silicon MLX build
$ ollama run gemma4:e4b-mlxCall Gemma 4 from the local Ollama API
$ curl http://localhost:11434/api/generate -d '{"model":"gemma4","prompt":"roses are red","stream":false}'Bake a system prompt into a custom Gemma 4 model
$ printf 'FROM gemma4\nSYSTEM "You are a terse coding assistant."\n' > Modelfile// llama.cpp & gguf
5 commandsRun a Gemma 4 GGUF straight from HuggingFace
$ llama-cli -hf ggml-org/gemma-4-E2B-it-GGUF -p "Write a haiku about the sea."Give llama.cpp a system prompt
$ llama-cli -hf ggml-org/gemma-4-E2B-it-GGUF -sys "You are Hong Gildong." -p "Who are you?"Start an OpenAI-compatible llama.cpp server
$ llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -c 8192 -ngl 99Use Google's QAT int4 weights for best quality per byte
$ llama-cli -hf google/gemma-4-E4B-it-qat-q4_0-gguf -p "Explain QAT in one sentence."Extend context length and offload layers to the GPU
$ llama-cli -hf ggml-org/gemma-4-12B-it-GGUF -c 32768 -ngl 99 -p "Summarize this file."// serve an api
5 commandsServe Gemma 4 with vLLM
$ vllm serve google/gemma-4-31B-it --max-model-len 16384Query a local OpenAI-compatible endpoint with curl
$ curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"google/gemma-4-31B-it","messages":[{"role":"user","content":"Explain MoE in one sentence."}]}'Enable tool calling and reasoning parsing in vLLM
$ vllm serve google/gemma-4-31B-it --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 --chat-template examples/tool_chat_template_gemma4.jinjaShard a large Gemma 4 across multiple GPUs
$ vllm serve google/gemma-4-31B-it --tensor-parallel-size 2 --gpu-memory-utilization 0.90 --max-model-len 16384Limit images and audio per request in vLLM
$ vllm serve google/gemma-4-E4B-it --limit-mm-per-prompt '{"image":4,"audio":1}'// prompting & control tokens
6 commandsFormat a Gemma 4 chat prompt by hand
Turn Gemma 4 thinking mode on or off
$ llama-cli -hf ggml-org/gemma-4-E4B-it-GGUF -sys "<|think|>You are a math tutor." -p "Solve 3x + 7 = 22."Control how long Gemma 4 thinks
$ llama-cli -hf ggml-org/gemma-4-E4B-it-GGUF -sys "<|think|>Think briefly, focus on key steps." -n 2048 -p "Is 2027 prime?"Keep thoughts out of multi-turn history
Put images and audio before the text
Use the recommended sampling settings
$ /set parameter temperature 1.0// python & transformers
4 commandsInstall Transformers to run Gemma 4 in Python
$ pip install -U transformers accelerate torch timmDownload a Gemma 4 checkpoint from the CLI
$ huggingface-cli download google/gemma-4-E4B-itLog in to pull Gemma 4 from HuggingFace
$ huggingface-cli loginLoad Gemma 4 in 4-bit to fit a small GPU
$ pip install -U bitsandbytes