vLLM 404 'Model Does Not Exist': Fix --served-model-name

Quick fix: call GET /v1/models and copy the exact data[].id into the request’s model field. If the name you want is not listed, fix --served-model-name or confirm that the client is pointing at the intended vLLM server. Do not redownload the weights just because the API returned a model-name 404.

A typical error looks like:

{
  "error": {
    "message": "The model \u0060my-model\u0060 does not exist.",
    "type": "NotFoundError",
    "param": "model",
    "code": 404
  }
}

The usual cause is simple: the model string in your request is different from the model name exposed by the server. The vLLM model registry checks the requested name against the names the server actually serves.

Step 1: Find out which 404 you have

An HTTP 404 has more than one possible meaning. First capture the response status and body instead of relying on the message shown by your application.

curl -sS -i http://127.0.0.1:8000/v1/models

If this endpoint answers with a valid model list, you are reaching an OpenAI-compatible vLLM endpoint. If it gives a proxy-style 404 or an unrelated HTML page, check the host, port, reverse-proxy path, and whether the intended server is running. A browser reaching some application at port 8000 does not prove it is reaching your vLLM instance.

Then send a minimal request to the expected API path:

curl -sS -i http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"local-chat","messages":[{"role":"user","content":"Say hello."}],"max_tokens":20}'

The model name in that request is an example. Replace it with the name shown in your actual model list. If the returned error says the model does not exist, continue to the alias check; if the server cannot be reached at all, fix connectivity first.

Step 2: Use the exact model ID returned by /v1/models

Request the registry’s model list:

curl -sS http://127.0.0.1:8000/v1/models

In the JSON response, inspect each item under data and its id. Use one of those exact IDs as the value of model in the subsequent completion request. Do not substitute a display label from your web UI, a local directory nickname, or the Hugging Face repository name unless the server actually advertises that exact string.

For example, a vLLM server might load Qwen/Qwen2.5-1.5B-Instruct but advertise the API alias local-chat. In that case, asking for Qwen/Qwen2.5-1.5B-Instruct can fail even though the weights loaded correctly. The registry matches the configured served names, not every plausible synonym.

The upstream model registry documentation is explicit: the check verifies whether the requested name is a served base model; an unmatched name receives a not-found response. That makes GET /v1/models the first reliable place to inspect a running server’s model identity.

Step 3: Understand –model versus –served-model-name

The vLLM CLI separates the model to load from the model identifier accepted through the API. Consider this example:

vllm serve Qwen/Qwen2.5-1.5B-Instruct \
  --served-model-name local-chat \
  --host 127.0.0.1 \
  --port 8000

Here the first argument identifies the model to load. The –served-model-name flag changes the name clients use in the model field. Your HTTP client should request local-chat, because that is the configured API name.

If you do not specify –served-model-name, the official CLI documentation says the served name defaults to the value of –model. The CLI also allows multiple served names; where multiple aliases are configured, each may be accepted while response metadata uses the first alias. Confirm what your installed version supports with its own help output rather than copying old command-line syntax blindly.

vllm serve --help

That help check is particularly useful when you run a packaged, pinned or vendor-modified build.

Step 4: Confirm your application uses the same endpoint

Even with the right model string, a client can point at a different service than the terminal you are testing. Look for configuration drift in four places:

1. Host and port. A local smoke test at 127.0.0.1:8000 does not prove that a container or remote application can reach the same address. Inside a container, 127.0.0.1 refers to that container.
2. Base URL. An OpenAI-compatible client generally needs its base URL to include /v1. If it already includes that prefix, do not append another /v1 in the request path.
3. Running process. The shell session that started vLLM may be using a different model, port or served alias from an older service process.
4. Proxy rewriting. A reverse proxy may route /v1/models correctly while forwarding chat requests to a different backend or stripping part of the path.

A good diagnostic captures one successful GET /v1/models and one failing request against the same base URL. Compare the IDs and timestamps. If your web UI lists one model but the API lists another, the UI is not authoritative for the server process.

Step 5: Do not confuse model-name errors with chat-template errors

A matched model ID does not mean every API operation is valid for the checkpoint.

vLLM’s chat endpoint requires an appropriate chat template for the model. The online-serving documentation explains that a model without a usable tokenizer chat template may fail on chat messages even while the server is otherwise reachable. When the model list is correct but chat requests fail for a template-related reason, fix the template or choose an endpoint appropriate for the model; changing the served model alias will not repair tokenization.

Similarly, a server starting to load weights but aborting before startup is a separate issue. Check startup logs, architecture support, GPU memory, tokenizer requirements, and the first reported exception. An absent or unhealthy server cannot be repaired by guessing an alias.

Minimal diagnostic record

When reporting a reproducible failure, retain the following details without sharing tokens or unrelated private data:

For an isolated local test, bind to the loopback address as shown above. Avoid publishing an unauthenticated inference endpoint to the public internet; routing and access control must be evaluated separately from model naming.

Decision checklist

If /v1/models itself fails, focus on process health and routing. If it succeeds but your requested model is not in data[].id, use a published model ID or change the configured served name. If the model ID matches and chat still fails, inspect the new error message for template or runtime problems instead of continuing to rename the model.

Primary sources

• vLLM serve CLI reference: –served-model-name
• vLLM model registry: model lookup and 404 response
• vLLM online serving: chat templates and API usage

For related diagnosis, see How to Run an LLM Locally and How to Reduce LLM VRAM Usage.