Serve Gemma 4 with vLLM
serve an api
// what it does
vllm serve loads a Hugging Face model and exposes an OpenAI-compatible server on port 8000 (/v1/chat/completions, /v1/completions). --max-model-len caps the context to keep KV-cache memory in check. vLLM's paged attention makes it the go-to for high-throughput, multi-user serving of the 26B and 31B sizes.
// shell
$ vllm serve google/gemma-4-31B-it --max-model-len 16384// gotcha
vLLM sizes the KV cache from --max-model-len at startup. Omit it and vLLM may try to reserve the full 256K context and fail with an out-of-memory error before serving a single request.