Query a local OpenAI-compatible endpoint with curl
serve an api
// what it does
vLLM, llama-server, and Ollama all speak the OpenAI /v1/chat/completions schema, so one curl works against any of them; only the port and model name change (vLLM 8000, llama-server 8080, Ollama 11434). The model field must match the served id. Add "stream": true for token-by-token SSE output.
// shell
$ curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"google/gemma-4-31B-it","messages":[{"role":"user","content":"Explain MoE in one sentence."}]}'// gotcha
The model string has to match exactly what the server loaded. For vLLM that is the full repo id google/gemma-4-31B-it, not a short alias, or the request returns a 404 model-not-found.