Run a Gemma 4 GGUF straight from HuggingFace
llama.cpp & gguf
// what it does
The -hf flag downloads a GGUF from a Hugging Face repo and runs it in one step, no manual conversion. ggml-org/gemma-4-E2B-it-GGUF hosts community builds of each size, and -p supplies the prompt for a single completion. Downloads cache under ~/.cache/llama.cpp so repeat runs start instantly.
// shell
$ llama-cli -hf ggml-org/gemma-4-E2B-it-GGUF -p "Write a haiku about the sea."// gotcha
-hf needs a repo that actually contains .gguf files. Pointing it at google/gemma-4-E4B-it (the raw safetensors repo) fails — use a *-GGUF repo such as ggml-org/gemma-4-E4B-it-GGUF or one of Google's QAT GGUFs.