Language model serving tool vLLM changes its default GPU deployment base in 0.33.0 and expands speculative decoding, which generates candidate responses in advance and verifies them before output. Developers serving language models must update their installation base when upgrading and can choose from more drafting and verification methods.

1. CUDA distributions now use 13.0 exclusively

According to the developers' release notes, CUDA wheels, the package files used for Python installation, and images are available only for CUDA 13.0. CUDA 12.9 distributions are no longer published, so developers using them must move to the new default distributions when upgrading to this version.

The base stack also moves to PyTorch 2.14.0 and Triton 3.8.0. This update changes related component versions alongside the new runtime base.

The default CUDA installation command is shown below, copied exactly from the release notes.

pip install vllm

ROCm 10.0 is now the default for AMD distributions. ROCm 7.2 remains available through separate images and wheels for users choosing the older base. Unlike the unified CUDA distributions, AMD retains two distribution paths.

2. More paths for refining and verifying drafts

In speculative decoding, a draft model proposes candidate next tokens, which the main model verifies. The newly added XPress refines these drafts. It covers block diffusion draft models such as DFlash and DSpark, which generate groups of tokens.

The Transformers modeling backend, the infrastructure responsible for running models, now supports multi-token prediction draft models known as MTP. This model execution path can now use drafting methods that predict multiple tokens.

For EAGLE and MTP, an improvement that retains the last block when reusing computation results for requests with the same prefix is enabled by default. This updates both drafting methods and the path for reusing existing computations.

3. Removing draft budget reservations boosts throughput

DFlash and DSpark no longer set aside draft slots, or preallocated generation space, from the token budget. This removes the behavior that reserved part of the request-processing budget for drafts in advance.

A benchmark for this change cited in the release notes used DeepSeek-V4.1-Flash with DSpark on GB300. Output throughput, the amount processed per unit of time, rose from 1373 to 3390 tokens per second. This shows that budget allocation changes, not just additional draft models, can improve performance.

Support was also added for quantizing the draft model's output layer, representing values with fewer bits. The release notes report an 18% decoding performance gain at 1 concurrent request with this lower-precision feature. Budget management and computation methods thus offer distinct improvement points within the same drafting path.

4. Added decoding optimizations for specific GPUs and models

GB300 users can opt for decoding optimized to reduce response latency. The developers reported 1.5–2x results for Kimi-K3 with model computation split across 8 or 16 GPUs. For DeepSeek-V4.1-Flash, they reported an 18.3% faster decoding stage at batch size 1.

Humming runs DeepSeek-V4/V4.1 experts stored in MXFP4, a low-precision numeric format, using block-wise FP8, an 8-bit floating-point format. Experts are model components selected according to the input. The release notes report 9–21% higher decoding performance with four H200 GPUs. This approach executes the model's internal expert computations in a different numeric format.

These performance figures come from specific hardware, models, and settings, so gains across different configurations are difficult to compare directly.

5. Review speed trade-offs alongside removed configurations

FLy is an optional policy that trades some verification rigor for speed. The developers report decoding throughput of 1.04–1.31x on MI355X and B300 in the release notes. It is optional so users can weigh higher throughput against the trade-off of changing verification methods.

The existing Medusa and MLP speculator drafting methods have been removed. While new drafting paths are expanding, services using these methods must select and implement replacements.

The existing Ray compiled graph executor, a tool that groups operations in advance for execution, has also been removed. Environments using it must move to remaining execution paths. Upgrade plans must therefore include replacing existing execution configurations as well as changing the deployment base.

vLLM 0.33.0 standardizes its deployment base on CUDA 13 and broadens speculative decoding improvements, so existing users should prepare to migrate installation and execution configurations and select suitable performance paths.