> prompting & control tokens
Gemma 4 uses the classic Gemma turn format — <start_of_turn>user ... <end_of_turn> then <start_of_turn>model — with no separate system role, so system instructions ride at the top of the first user turn. On top of that it adds a trained-in thinking mode toggled with the <|think|> token (or enable_thinking in apply_chat_template), which emits a hidden chain of thought before the answer. Libraries usually apply the template for you; these entries cover the pieces you tune by hand.
// prompting & control tokens
6 commands$ llama-cli -hf ggml-org/gemma-4-E4B-it-GGUF -sys "<|think|>You are a math tutor." -p "Solve 3x + 7 = 22."$ llama-cli -hf ggml-org/gemma-4-E4B-it-GGUF -sys "<|think|>Think briefly, focus on key steps." -n 2048 -p "Is 2027 prime?"$ /set parameter temperature 1.0// faq
How do I enable thinking mode in Gemma 4?
Put the <|think|> token at the start of the system instruction, or pass enable_thinking=True to apply_chat_template. The model then reasons inside a <|channel>thought ... <channel|> block before its final answer. Remove the token to disable it. Thinking helps on math, proofs, coding, and agentic tasks and adds latency, so skip it for simple lookups.
Why is Gemma 4 emitting empty thought tags even with thinking off?
On the 12B, 26B, and 31B, an empty <|channel>thought<channel|> pair is deliberately added to the chat template to suppress stray 'ghost' thought channels when thinking is disabled. E2B and E4B skip it. It is harmless, but your response parser should tolerate the empty block rather than treat it as an error.
What are the recommended sampling settings for Gemma 4?
Google recommends temperature 1.0, top_p 0.95, and top_k 64. The temperature of 1.0 is higher than the 0.7 many other models use, so copying a low-temperature preset from a different model can make Gemma 4 feel flat or repetitive. Lower the temperature only for deterministic extraction or classification.
Should I keep the model's thoughts in conversation history?
No. Strip the <|channel>thought ... <channel|> block from previous turns and pass back only the final response before the next user turn; carrying thoughts forward wastes context and can derail reasoning. The exception is tool-call turns, where the thinking that produced the call should be preserved.
Does the order of images and text in a prompt matter?
Yes. Place image and audio parts before the text instruction. Gemma 4 is trained with media tokens ahead of the question, so a text-first prompt still runs but produces measurably worse captions, OCR, and answers — an easy problem to misdiagnose as poor model quality.