vLLM guided_json Removed: Fix Structured Outputs Requests

Quick fix: on current vLLM, replace legacy guided_json, guided_regex, guided_choice, and guided_grammar request fields with structured_outputs. Remove the old per-request guided_decoding_backend field.

If the request then fails with Failed to transform json schema into a grammar, the API migration is no longer the problem; isolate the JSON Schema or grammar backend instead.

First check the installed vLLM version

The current vLLM structured outputs guide states that legacy guided-decoding request fields were removed in v0.12.0. Do not assume a sample written for a much older server still matches a newer API.

python -m pip show vllm
vllm serve --help

If the server runs in a container, check the version inside that container, not only in the Python environment used to send the request. A client package version and a remote server package version are separate facts.

Replace the legacy request body

Old client examples commonly used a top-level extension like this:

extra_body={"guided_json": {"type": "object"}}

For the newer API, pass the constraints under structured_outputs:

extra_body={"structured_outputs": {"json": {"type": "object"}}}

The upstream migration map is:

For instance, the current documentation demonstrates the standard OpenAI-compatible client with a vLLM-specific extra_body:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="-")
model = client.models.list().data[0].id

schema = {
    "type": "object",
    "properties": {"status": {"type": "string", "enum": ["ok", "fail"]}},
    "required": ["status"],
    "additionalProperties": False,
}

result = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Return JSON with status ok."}],
    extra_body={"structured_outputs": {"json": schema}},
)
print(result.choices[0].message.content)

This example requires an already running, compatible local vLLM service and the OpenAI Python client. It is a request template, not a guarantee that any particular model will produce semantically correct content. Confirm the model ID returned by your server and validate the JSON downstream.

What if the JSON Schema is still rejected?

Changing the request field does not make every JSON Schema feature supported. vLLM can turn a schema into a constrained-decoding grammar through XGrammar. Its XGrammar backend reference documents two distinguishable error categories:

• The provided JSON schema contains features not supported by xgrammar. Some constructs are not accepted by the selected backend.
• Failed to transform json schema into a grammar. The backend attempted conversion and the conversion failed, with an underlying error.

Start with a deliberately simple schema using object, primitive properties, required fields and an enum. If it succeeds but the original does not, add schema constructs back one at a time. Reduce nested unions, exotic constraints or generated schema features only if the conversion error points in that direction; do not delete validation rules blindly in production.

Also verify that the object passed to the JSON field is a real JSON Schema, not a whole chat-completions response_format wrapper accidentally embedded as the schema. If you use response_format, follow the separate example in the official guide instead of mixing the two request shapes.

Backend selection and version drift

The current structured outputs configuration reference describes auto as the default backend selection, with alternatives depending on version and supported grammar features. An error under one selected backend does not imply all backends behave identically.

The feature guide documents server configuration through the structured-outputs backend option. Inspect the installed vllm serve –help before copying a flag. Avoid using a deprecated per-request guided_decoding_backend field to force a backend that the newer API no longer accepts.

Changing backends can alter supported schema features, regex syntax and output behavior. Test the exact request after any change. A successful request with constrained syntax is not proof of factual accuracy or downstream business-rule validity.

A diagnosis matrix

Preserve the HTTP status, response body, server version, redacted request schema and the exact error text. A client-side timeout without a returned error is not equivalent to a rejected schema.

Do not use weaker constraints as a silent fix

For development, simplifying the schema is a valid way to localize a failure. For production, a schema downgrade changes the interface contract. If a required property, type or business rule was removed for compatibility, document that reduction and enforce equivalent validation downstream.

Likewise, do not treat constrained generation as validation of external facts. A model can produce syntactically valid JSON whose values are inaccurate. Treat structural validity, content correctness and authorization as different checks.

Verify the repair

1. Identify the live server version.
2. Replace removed request fields with the documented form.
3. Run a minimal schema and preserve the full response.
4. Add original schema features back until the problematic construct is isolated.
5. Validate the returned JSON against the intended full schema in the application.
6. Retest the actual model, sampling settings and client wrapper used in your deployment.

For a different vLLM integration failure involving tool calls rather than constrained output, see the tool-call parser and template guide.

Primary sources

• vLLM: Structured Outputs and migration
• vLLM: XGrammar backend validation
• vLLM: Structured Outputs configuration