> serve an api
For production or multi-user serving, vLLM loads Gemma 4 and exposes an OpenAI-compatible server on port 8000 with paged-attention throughput, tensor-parallel sharding across GPUs, and parsers that split out tool calls and thinking. Ollama and llama-server speak the same /v1 schema for lighter setups, so one curl or SDK client works against all three — only the port and model name change.
// serve an api
5 commands$ vllm serve google/gemma-4-31B-it --max-model-len 16384$ curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"google/gemma-4-31B-it","messages":[{"role":"user","content":"Explain MoE in one sentence."}]}'$ vllm serve google/gemma-4-31B-it --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 --chat-template examples/tool_chat_template_gemma4.jinja$ vllm serve google/gemma-4-31B-it --tensor-parallel-size 2 --gpu-memory-utilization 0.90 --max-model-len 16384$ vllm serve google/gemma-4-E4B-it --limit-mm-per-prompt '{"image":4,"audio":1}'// faq
How do I serve Gemma 4 with vLLM?
Run vllm serve google/gemma-4-31B-it --max-model-len 16384. It exposes /v1/chat/completions on port 8000. Always set --max-model-len, because vLLM sizes its KV cache from it at startup and can otherwise try to reserve the full 256K context and run out of memory before serving.
How do I enable tool calling and reasoning output in vLLM?
Add --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 and the Gemma 4 tool chat template. The parsers move tool calls into tool_calls and the model's thinking into a separate reasoning_content field. Without the reasoning parser, the raw thought block stays inside the message content and leaks to users.
Can I run the 31B across multiple GPUs?
Yes. --tensor-parallel-size N shards the weights across N GPUs, so a bf16 31B (~58 GB) fits on two 40 GB cards. The value must divide the attention-head count evenly and the GPUs should be identical. Pair it with --gpu-memory-utilization to leave headroom for other processes.
Which Gemma 4 sizes accept audio when serving?
Only E2B, E4B, and 12B have an audio encoder; the 26B and 31B are text plus image. In vLLM, set the audio key in --limit-mm-per-prompt only for the audio-capable sizes, since it has no effect on the image-only 26B and 31B.