Qwen3.8-27B DSpine: Layer-wise conditioning, parallel drafting

1. Passing earlier predictions to later computations

DSpine is a draft model that proposes candidates for Qwen3.8-27B to verify. In speculative decoding, an auxiliary model drafts candidates and the target model checks them together. An incorrect early candidate discards later ones, making consistency important.

At each layer, prediction information passes to the next position, whose receiver controls how much to incorporate while all positions compute in parallel. Token selection uses a transition cache of precomputed scores for neighboring candidate pairs to reduce sequential computation.

2. A public model for use with Qwen3.8-27B

Tencent's released checkpoint, a set of trained model files, is a five-layer draft model for Qwen3.8-27B. Its block size is 16, and its sliding window, the span of recent context consulted at once, is 8,192 tokens.

The BF16 weights contain about 2 billion parameters, occupy about 4.05GB, and use the Apache 2.0 license. Deploying it alongside the target model also requires these auxiliary weights.

The model card provides this installation example.

git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .

After installation, specify the target and draft model paths in SGLang to run them together. The target model generating responses stays unchanged, with an auxiliary model attached to generate candidates.

3. Acceptance lengths reported in the model card

Acceptance length is the average number of tokens advanced per verification round. Tencent reports longer acceptance lengths for DSpine than DFlash2 on the five tasks below at temperature 0. Their respective block sizes are 16 and 8.

Table: Acceptance lengths on general benchmarks at temperature 0

Conditions: Qwen3.8-27B, SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8, xhigh reasoning mode, up to 4,096 new tokens, temperature 0; DSpine block 16, DFlash2 block 8; DSpine uses dynamic verification and ReplaySSM

Unit: Average tokens/round

Acceptance-length differences varied by task even in the same runtime environment. Tencent also reports longer acceptance lengths on all five tasks in sampling evaluations at temperature 1.

4. 3.84x autoregressive throughput in office-task replay

Tencent reports DSpine's end-to-end throughput at 3.84x autoregressive generation in its office-task evaluation. Autoregressive generation produces tokens sequentially based on preceding tokens. Conversation logs, including tool calls, were replayed in order. Throughput divides total output tokens by the total elapsed time of concurrent replay.

Table: End-to-end throughput in office-task replay

Conditions: WorkBuddy Office, Qwen3.8-27B, SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8, low reasoning mode, temperature 1.0, cumulative probability cutoff 0.95, sampling from the top 20 candidates, 20 conversations. Logs were replayed in CodeBuddy's sandbox, an environment isolated from external systems. Prefix caching reuses earlier computation results. Up to 32,768 new tokens per request; DSpine block 16, DFlash2 block 8; DSpine uses dynamic verification and ReplaySSM.

Under these conditions, better acceptance length also translated into higher throughput. Tencent reports that DSpine achieved longer acceptance lengths than comparison models on other agent tasks in the model card as well.

5. More candidate computation for better verification efficiency

In a separate paper, the researchers report that Qwen3-8B's average acceptance length at temperature 0 rose from DFlash's 3.77 to 4.82 with the new model. This averages seven tasks with reasoning mode off and 15 candidates proposed per round.

The paper also reports average SGLang throughput 23.3% higher than DFlash. Measurements covered four math and coding tasks.

This serving evaluation used a single GPU, BF16, FlashAttention-3, temperature 0, concurrency 2–32, and up to 2,048 output tokens. Different target models and runtime conditions make direct comparisons with the model card difficult.

The gain comes from confirming more tokens per verification. In the paper's Qwen3-8B latency analysis, verification took 72–76% of round time. The new model's round cost was 7.14 milliseconds, similar to DFlash2's 7.19 milliseconds. Decoding time per generated token was 10.9% lower than DFlash2.

The trade-off is extra auxiliary-model computation. Layer-wise propagation added up to 0.15 milliseconds per round across five layers. The transition cache also added computation to prescore candidate pairs. Longer candidate acceptance offset these costs.

DSpine's value lies not just in generating candidates quickly, but in confirming more tokens per verification to offset additional computation.