vLLM 0.31.0 introduced vllm preload on October 5, 2026. It keeps processed GPU weights available for a restarted engine. Implementation details here refer to the v0.31.0 release tag.

Keeping processed weights across an engine restart

The cost of bringing a serving engine back up sits before any kernel runs. Every restart repeats reading the checkpoint from disk, quantization post-processing and sharding across TP ranks. The vLLM documentation says preload cuts that restart time from "minutes" to "seconds", but gives no model, GPU or measurement conditions.

The structure is simple. `vllm preload` starts one daemon process per GPU; each daemon loads its rank's shard once, finishes quantization post-processing and exports every parameter as a CUDA IPC handle. An engine started with `--load-format ipc_cache` builds only the model skeleton on the storage-less meta device and requests tensors from the daemon on the same GPU over a Unix domain socket.

In default zero_copy mode, the engine maps the daemon’s GPU memory directly, so there is only one weight copy per GPU. The daemon must stay alive, and weight-offloading sleep mode is incompatible. The alternative copy mode creates an engine-owned copy but releases the daemon cache after the handoff, trading reuse for independent ownership.

Why matching safetensors headers is insufficient

Before mapping, the engine compares a fingerprint (WeightCacheKey) with the daemon. The documentation lists checkpoint contents, model architecture, TP/DP size and rank, dtype, quantization method and config, model revision and vLLM version. Its "Safe fallback" description says any difference sends the engine back to disk loading.

The question is what "checkpoint contents" covers. Reading `protocol.py` at the v0.31.0 tag shows that `hash_checkpoint()` feeds only each shard's file name and safetensors header into the hasher. A header holds tensor names, dtypes, shapes and byte offsets, not weight values. The docstring states plainly that the design was meant to give the same key to identical weights stored in different directories.

Issue #59647, filed on October 1, reproduced the side effect. The reporter copied Qwen3-0.6B, left the header untouched and overwrote only `model.norm.weight` with zeros. Served on its own, the copy printed "!!!!!!!!!!!!", but attached via `ipc_cache` to a daemon holding the original, it produced the same sentence as the original. Neither process logged a warning.

According to the report, the hashed header in Qwen3-0.6B was 35,560 bytes out of a 1,503,300,328-byte file. The fallback path itself works when fingerprints disagree. Issue #60178 shows an engine logging a warning and falling back to disk when `dp_size` differed under data parallelism on an October 3 main build. The gap is a single fingerprint that never looks at values.

Fix PR #59648 mixes each shard's file size and the first 1 KiB of every tensor's data region into the fingerprint. The PR itself notes the limit that the key does not move if values change only past the first 1 KiB of every tensor, and defines it as a cache-correctness fingerprint rather than a security boundary. Both the PR and the original issue were open when checked on October 14, 2026.

Sixty-six header reads on every restart

The same fingerprint also creates a cost in the opposite direction. In v0.31.0 `ipc_loader.py`, `_fetch_entries()` calls `WeightCacheKey.from_model_config()` every time it requests tensors from the daemon, and that function runs `hash_checkpoint()` again. Even when the daemon already holds the weights, the original checkpoint files are opened on every restart.

One user measured this cost in issue #60944. The setup was vLLM 0.31.0, Qwen3.8-27B-FP8 in 66 shards totaling about 28.75 GiB on OSSFS, a single H20-3e and TP=1. With the original fingerprint computed every time, reaching `/health` 200 took 78.79 seconds on average. With a change that reuses the daemon's first fingerprint, it took 46.85 seconds. Both runs used the same daemon, `zero_copy`, `fallback=false` and a prewarmed compile cache.

Breaking the numbers down shows what kind of cost this is. In the user's instrumentation posted on PR #59648, reading headers took about 32 seconds for 66 files totaling 203,664 bytes. Simple arithmetic gives about 0.49 seconds per shard for a little over 3 KB read per file. Round-trip latency of the remote filesystem, not bandwidth, ate the restart time.

Even with the fingerprint skipped, the model-loading phase took about 5.6 seconds while the whole startup stayed around 47 seconds. The documentation's "seconds" reads as a description of the weight-loading phase. Skipping `profile_run` on a cache hit and measuring CUDA graph capture time remain unfinished items on the Fast Start roadmap.

The result comes from one setup, not a general speedup claim. Local NVMe may avoid the same delay, but no corresponding measurement was published. A separate proposed fix groups tensor-sample reads into 32 concurrent requests; its remote-storage startup cost was not measured in this report.

When a cache miss quietly reloads from disk

The default `fallback: true` makes failures quiet. According to issue #60178, reported against a source build of main from October 3, serving a dense non-MoE model with `--data-parallel-size 2` makes the daemon's and engine's `dp_size` fingerprints disagree. The server answers normally, but each rank logs one warning line and reloads from disk. Meanwhile the daemon kept holding its own copy on every GPU.

`fallback: false` surfaces such misses as startup failures. A header collision like #59647, however, is a case where the fingerprints match, so this setting does not catch it. Misses are buried in warnings, and wrong hits leave no signal at all.

The roadmap in Fast Start feature issue #56049 records the same gaps. Two items were still unfinished when checked on October 14. One is "Daemon-loss semantics": a running engine's weights become invalid once the daemon exits. The other is output-equivalence tests in CI that compare greedy tokens from cached and regular loading.

A different trade-off from SGLang weight loading

SGLang solves a related but different problem: its memory-saver mode releases GPU memory without ending the server process, and its update_weights APIs replace the running model’s weights. The remote_instance loader can also fetch weights from another active SGLang server.

The referenced SGLang documentation does not describe a checkpoint-equivalence check for remote_instance. vLLM preload instead starts a new engine and borrows another process’s GPU weights, making a correct cache fingerprint essential to selecting the intended checkpoint.

The benefit of preload therefore depends heavily on operating conditions. In a deployment where one daemon keeps serving a single checkpoint from an immutable model store, the gain from skipping disk loading and quantization post-processing stays intact.

Repeated fine-tuned checkpoints or reinforcement-learning snapshots are a riskier fit: the same tensor shapes can conceal different values, precisely the mismatch demonstrated in issue #59647.

Checking daemon health and cache reuse

v0.31.0 `preload.py` has `--weight-cache-health-port` for the daemon parent process, and `/health` returns 200 once every rank is ready and 503 before that or when a rank has died. The port is off by default, and when enabled its default host is 0.0.0.0.

On the engine side, `fallback: false` turns a miss into a startup failure. The next step is to see which line the engine log shows: the mapping-success line or the fallback warning. Both strings come from the v0.31.0 `ipc_loader.py` log formats and the issue reproduction logs.

vllm preload --model <local-model-dir> --weight-cache-health-port <port>
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:<port>/health
vllm serve <local-model-dir> --load-format ipc_cache \
    --model-loader-extra-config '{"fallback": false}'
# engine log
#   "Mapped <N> tensors from the weight cache daemon (zero_copy mode)"
#   "Weight cache unusable (...); falling back to disk loading"

A mapping-success line does not guarantee the right weights. The remaining check in 0.31.0 is the one the #59647 reporter used: right after switching checkpoints, temperature 0 output is compared once against an engine serving the same model with the default load format (`auto`).

The next change will likely come from tuning how much the fingerprint reads. #59648 reads more per-tensor samples for accuracy, while #60944 proposes an opt-in mode in which the operator explicitly trusts a specific daemon and skips the recheck. Which default these two directions settle on will probably decide whether preload can be attached to training loops.

The restart saving in vLLM 0.31.0 is real in the documented cache path, but a header-only fingerprint cannot prove that the reused weights belong to the requested checkpoint.