SGLang, a tool for developers serving language models, adds Mac execution options and improves model loading and disaggregated server throughput. Local development and large-scale serving gain options, but performance gains depend on storage and request mix.

1. Macs gain a standard PyTorch path

Released on October 19, 2026, v0.5.22 adds a standard PyTorch path on Apple Silicon alongside the existing Mac-specific path. Macs can now test scheduling and model execution code shared with other accelerators.

The new path does not support quantized models, models accepting non-text inputs, or multi-device execution. Speculative decoding, which predicts and verifies upcoming tokens, and disaggregated serving are also excluded.

Mac CPUs and GPUs share memory, so cache sizing uses the operating system's recommended budget rather than total physical memory. The Mac validation ran Qwen3-0.6B on an M4 Pro with 48 GB, allocating 0.6 of that budget to static memory. Accuracy was 0.425 on 200 math questions with 5 examples and concurrency of 8.

Online serving on the same hardware and model handled 213.49 combined input and output tokens per second. The test used 40 random requests, 256 input tokens, 128 output tokens, and a maximum concurrency of 8. This validates Mac execution, not superiority over other accelerators.

2. Model loading depends on storage conditions

InstantTensor, an optional loader, reads weights in chunks and prefetches upcoming data. It retains the existing file discovery process and is selected after a separate installation, as shown below.

Supported devices are CUDA and ROCm; the new Mac execution path is not supported. Only files in Safetensors format are supported.

pip install "sglang[instanttensor]"

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-4B \
  --load-format instanttensor \
  --model-loader-extra-config '{"backend":"URING","chunk_size":8388608,"io_depth":64}'

In the loading comparison, reading DeepSeek-R1 weights with 50 GB/s storage and 8 H200 GPUs took 15.3 seconds versus 160 seconds with the existing loader, a 10.5-fold speedup. With memory-backed storage, the same model and hardware took 12.1 seconds versus 22.6 seconds, a 1.87-fold speedup. These are weight loading times, not total times until the server is ready.

Loading Qwen3-30B-A3B on 1 H200 with 50 GB/s storage took 57.4 seconds with the existing loader and 1.77 seconds with the new loader. Different models and device counts prevent a direct comparison with the preceding results.

3. Disaggregated serving changes when memory is reserved

Disaggregated serving assigns bulk input processing and sequential token generation to separate servers. The new optional policy delays reserving the decode server's KV cache, which stores intermediate computations, until input processing finishes. It reduces space reserved by long requests that cannot yet generate tokens, rather than accelerating computation itself.

The earlier disaggregation policy experiment used GLM-5.2-NVFP4, 12 Rubin GPUs, and 3-step multi-token prediction. It split and replicated model computation with 2 prefill replicas and 1 decode replica, setting tensor, data, and expert parallel sizes to 4 each.

The experiment deliberately constrained cache space on the decode server.

> Test conditions: 8-bit floating-point cache, Mooncake transfer, and 163,840 decode cache tokens per device. Both policies used 1 early prefill attempt.

Each policy was measured twice with 64 requests of 131,072 input tokens and 192 requests of 4,096 input tokens, with 256 output tokens and concurrency of 64. Comparing arithmetic means of per-run metrics, throughput rose 26.81% and the 95th percentile of time to first token fell 48.87%. The 95th percentile is the point at or below which 95% of measurements fall.

However, mean time per output token rose 2.13%, and the 99th percentile of inter-token latency rose 92.77%, increasing pauses at the slow end.

Other costs include retaining the prefill server's cache longer and losing overlap between input processing and cache transfer.

For smaller models, the throughput difference was within run-to-run variation.

Initial support covers configurations using Mooncake transfer. Apart from splitting overall model computation across four devices, supported configurations do not further split attention, input context, or model stages across devices. Attention computes relationships between tokens.

> Deployment check: Upgrade communication workers with the default policy first, then switch the policy for both roles together.

These results predate the baseline code update, and regression validation against the latest code remains incomplete in the proposed change.

4. Distinguish hardware support from performance comparisons

The release notes report a 2.1~2.3-fold increase in output throughput per GPU for DeepSeek-V4.1 Flash on 8 H200 GPUs. Input lengths and request concurrency were not provided.

On one of the two supported devices, speculative decoding throughput measured up to 1.9 times that of regular decoding.

> Hardware and configuration check: AMD MI350X and MI355X are supported. Measurements used 4 MI350X GPUs and DSpark.

With DSpark in this configuration, time to first token for 128K input fell from 860 milliseconds to 576 milliseconds. The same performance has not been confirmed on every supported device.

5. Check Python before upgrading

Python 3.11 or later is required. On 3.10, installing without specifying a version selects the last compatible release, so a successful installation alone does not confirm an update.

This release expands Mac development paths and server optimizations, but the policy suits mixed input lengths and limited decode cache; throughput alone is insufficient for services sensitive to long response pauses.