NVIDIA has released Snapshot, an open-source tool for Kubernetes. It saves the state of workloads running in initialized GPU environments and restores it in new instances. Inference teams can reduce model preparation time, accelerate recovery, and respond faster to demand.
1. Reuse execution state that is ready to run
Snapshot is part of the NVIDIA DSX AI Factory Platform and saves execution state once preparation is complete for later restoration. It stores CPU and GPU memory as portable file bundles called artifacts for reuse in new tasks. This avoids repeating preparation steps such as reloading model weights and optimizing execution.
Originally developed for NVIDIA Dynamo inference workers, it can also integrate with other inference engines, schedulers that assign workloads, controllers that reconcile requested state, and platforms.
Compatible workloads using multiple GPUs on one node are also saved as a single artifact. This supports large models that use parallelism to distribute computation across multiple GPUs.
2. Create request objects to save and restore state
Checkpointing, which saves execution state, and restoration use dedicated request objects in Kubernetes. Create a PodSnapshot to save a running workload, or use a SnapshotJob to start from a template and save it as soon as it is ready.
To restore, add the nvidia.com/restore-from annotation to a new Pod to specify the source checkpoint.
Workloads must be integrated to pause briefly just before checkpointing and resume after restoration. NVIDIA's repository provides vLLM, SGLang, and TensorRT LLM integration examples that platform teams can adapt to existing inference environments.
3. B200 tests show restores of 3.6~42.7 seconds
NVIDIA reported tests using NVIDIA B200 GPUs, driver 615.71.09, and VAST(NFS) storage. The results below use the same model on one GPU, comparing each engine's cold start from scratch with restoration from saved state.
Cold start, which initializes the execution environment from scratch, was measured from process launch to engine initialization completion. Restoration was measured from the request to service resumption.
Table: Qwen3-0.6B cold start versus Snapshot restore
Conditions: NVIDIA B200, driver 615.71.09, VAST (NFS), Qwen3-0.6B, TP1(1 GPU)
Units: time in seconds, speedup in multiples
| Inference engine | Cold start (seconds) | Restore (seconds) | Speedup (x) |
|---|---|---|---|
| vLLM 0.31.0 | 104.43 | 4.66 | 22.40 |
| SGLang 0.5.21 | 87.08 | 3.59 | 24.26 |
| TensorRT LLM 1.3.0 | 121.26 | 8.15 | 14.88 |
In the B200 and VAST storage results above, all three engines restored faster than they cold-started. Across all configurations reported by NVIDIA, restores took 3.6~42.7 seconds, 8~36x faster than cold starts. Large models using 8 GPUs restored checkpoints of about 600GB~1TB in under 43 seconds, versus cold starts of 6~19 minutes for those configurations.
Design partner Crusoe said it cut GLM-5.3 Pod readiness time on NVIDIA HGX B300 from about 8 minutes to 31 seconds. Saturn Cloud also reported reducing vLLM startup on NVIDIA HGX B300 from 2~8.5 minutes to 6~34 seconds.
4. GPU state transfer drives restore time for large models
Cold start involves engine startup, weight loading, memory preparation, and kernel compilation, autotuning, and graph capture. NVIDIA reported that the final compilation, tuning, and capture stage took 784.54 seconds for DeepSeek-V4.1-Flash with vLLM, out of 1,110.65 seconds for the full startup. Snapshot saves completed initialization state to skip this repeated work.
Restoration locates the artifact, restores CPU process state and GPU memory, then resumes execution. CPU state is read using CRIU, a Linux process checkpoint and restore tool.
NVIDIA reported GPU state transfer times of 18~29 seconds and CPU state restoration times of 4.2~10.8 seconds when restoring large models on 8 GPUs. Reducing large-model restore time requires examining the data transfer back to GPUs.
5. Shorter startup requires checkpointing work and storage
NVIDIA reported checkpoint creation times of 15 seconds~5.5 minutes and artifact sizes of 8.04GB~1,039.09GB, depending on configuration. Creation is a one-time cost for each model and configuration. For every tested configuration, checkpointing plus the first restore took less time than one cold start. Reducing repeated initialization means accounting for checkpointing work and large artifacts in operations.
Measured times exclude container image downloads and container runtime preparation. Restoration requires the same GPU model and driver, plus execution compatibility. The KV cache, which temporarily stores conversation context and is cleared before checkpointing, is not included in artifacts.
Snapshot reduces startup time by reusing initialized GPU workload state, while effective adoption requires accounting for checkpoint time and storage capacity.