vLLM v0.33.0 Upgrade Audit: CUDA 13 Only and Removed Features
A source-led audit of vLLM v0.33.0 breaking changes: CUDA 12.9 artifacts, removed Medusa and MLP speculators, the Ray executor, NIXL 1.5 and stricter API behaviour.
Approximately 6 min read
vLLM v0.33.0, published on November 2, 2026, is easy to misread as a model-support release because its notes list eleven new architectures. The more important content for anyone who already runs vLLM sits in the platform and Breaking Changes & Deprecations sections. According to the official release notes, the release contains 927 commits from 389 contributors, moves to PyTorch 2.14, and publishes CUDA 13.0 artifacts only.
This article does not benchmark the release. It converts the project’s own notes into an upgrade audit: what the notes say changed, which deployments are plausibly affected, and what you should verify in a staging environment before you move production traffic. Every behavioural statement below is the release notes’ claim, not something we measured.
What the release notes state
The platform baseline is PyTorch 2.14.0 with torchvision 0.29.0 and Triton 3.8.0 (PR #52183). The notes say CUDA 12.9 wheels and the -cu129 images are no longer published (#62575), and that the CUDA wheel is now ABI stable with libtorch. They also say every image is additionally published under the shorter vllm/vllm name, and add an AWS EFA image, vllm/vllm-openai:v0.33.0-efa, for x86_64 and aarch64 (#58768).
The removals and deprecations are listed explicitly:
- Removed: CUDA 12.9 wheels and images; Medusa and the MLP speculator (#61701); the legacy Ray compiled-graph executor (#61712, #62228), including the
VLLM_USE_RAY_V2_EXECUTOR_BACKENDvariable, soRayExecutorV2is the only Ray executor. - Deprecated:
--mamba-cache-mode, which now has no effect (#60043);--ir-op-priority, with removal planned for v0.34.0 (#63371); and the LM Format Enforcer structured-output backend, which becomes opt-in in v0.34.0 and is slated for removal in v0.35.0 (#63423). - Connector requirement: the NIXL connector needs NIXL 1.5 or newer (#61032).
API behaviour changes that can break clients
Several items change what a running server returns, which matters even if your build and driver stack is unaffected:
- Requests sent while generation is paused now return HTTP 503 (#51488). Reinforcement-learning or weight-update workflows that pause generation and still send requests should be tested for retry handling.
kv_transfer_paramsandec_transfer_paramsmoved to the top level of render output (#61241). Any code that parses these fields from the old location needs updating.- LoRA requests against a model without LoRA support now fail at validation (#61884), and beam search scores are normalized by generated length (#62928). The second change can alter ranking in applications that compare beam scores across runs.
- For the Rust frontend, unsupported
vllm serveflags and request fields now fail instead of being silently ignored (#62431, #62434). Request-supplied chat templates require--trust-request-chat-template(#62673), and an emptytoolsarray returns HTTP 400 (#63060).
The Rust-frontend items deserve special attention because “silently ignored” is a common property of long-lived launch scripts. A flag copied between versions for years may have been doing nothing; after this release it can prevent the server from starting.
Speculative decoding: removal and improvement in the same release
The release both deletes older drafting methods and improves newer ones. Medusa and the MLP speculator are gone; the notes do not name a replacement, but they list continued work on EAGLE, MTP, DFlash and DSpark paths, including XPress (#54448), the FLy lossy verification policy (#53987), quantized draft lm_head (#59973) and lookahead block hashes enabled by default for EAGLE/MTP (#61589). DFlash and DSpark no longer reserve draft budget slots (#59468).
Two details are worth treating carefully. First, FLy is described as a lossy verification policy; a lossy verifier trades exactness for speed, so it should not be enabled for a workload that needs outputs identical to non-speculative decoding without a separate equivalence test. Second, the notes report that skipping drafts no request can use gives up to a 7.3% prefill throughput gain under DFlash (#62784). That figure is the project’s claim for its own test configuration and should not be assumed to transfer to your model, batch shape or hardware.
If you serve a Medusa or MLP-speculator checkpoint today, there is no in-release fallback: you either pin to an earlier vLLM or move to one of the speculative methods still receiving changes in this release, which means a different checkpoint and a fresh acceptance-rate evaluation.
Hardware and performance claims
The notes include several large hardware-specific numbers: FlashInfer DCP prefill with 39–48% lower mean time to first token at 100k input on 8×B200 (#62029), latency-specialized decode for Kimi-K3 on GB300 (#61867), and a ROCm MonoKernel decode path enabled with --monokernel. On AMD, ROCm 10.0 becomes the default image and wheel target, with ROCm 7.2 kept as -rocm72 (#58761), and three ROCm environment variables were removed while VLLM_ROCM_MONO_DECODE was replaced by --monokernel.
These are source claims. They describe specific models, GPUs and settings, usually on hardware most self-hosters do not have. A responsible reading is: there are plausible gains on those paths, not your deployment will be faster.
Security notes worth acting on
The release notes list per-request budgets for downloaded and decoded media bytes (#61226), a restriction that speculators checkpoint metadata can no longer reference local verifier paths (#61587), certifi>=2026.10.19 with refreshed CA certificates (#62868), and raised minimum versions of aiohttp, Pillow, MCP and multidict (#62835). If your image build pins any of those packages, check for conflicts with the new minimums.
A staged upgrade checklist
- Inventory the stack below vLLM. Confirm your NVIDIA driver supports CUDA 13 using NVIDIA’s own compatibility documentation, and rebuild or re-pin anything compiled against CUDA 12 or an older PyTorch, such as custom kernels and extensions.
- Grep your launch configuration. Search scripts, Helm values and compose files for
--mamba-cache-mode,--ir-op-priority,VLLM_USE_RAY_V2_EXECUTOR_BACKEND, the removed ROCm variables, and any Medusa or MLP-speculator model reference. - Check connectors. If you use NIXL for disaggregated serving, verify that every node runs NIXL 1.5 or newer.
- Replay real traffic against staging. Capture a representative set of requests, including tool calls with and without
tools, LoRA requests, and paused-generation flows, and compare status codes and payload shapes with the old version. - Treat speculative decoding as a new configuration. Measure acceptance rate and end-to-end latency on your own prompts. For any lossy policy, run a quality comparison, not only a speed test.
- Plan for the next two releases. LM Format Enforcer and
--ir-op-priorityare already scheduled for change in v0.34.0 and v0.35.0, so migrating structured-output usage now avoids a second upgrade fire drill. - Keep a rollback. Because CUDA 12.9 images no longer exist for this version, rollback means running the previous version’s image, not a different build of the new one.
What this audit does not establish
We did not install v0.33.0, run its tests, or compare throughput. We read one primary source: the project’s release notes. That page contains the maintainers’ summary, not an independent verification of each item, and the tag it is published under (v0.33.0rc3) reflects how the project cut the release. For any item that affects you, open the linked pull request and read the diff and discussion; the PR numbers above are given for that purpose.
The practical conclusion is narrow but firm: v0.33.0 is a platform migration first and a feature release second. Teams on CUDA 12.9, Medusa or MLP speculators, the legacy Ray executor, or older NIXL builds should schedule the upgrade as a planned change with a staging replay, not as a routine pip install -U.