NVIDIA has released a 4-billion-parameter language model that reuses the same computational layers. Developers deploying small models can adjust loop counts to balance task-specific accuracy and compute without retraining or replacing the model.

1. Adjust loops rather than model size

Released on October 16, 2026, Dory1.0-4B repeatedly processes intermediate representations inside the model. It passes them through the same computational block multiple times, reinjecting the input on each loop.

NVIDIA recommends 2 loops as a starting point for balancing accuracy and compute. It suggests 1 to minimize compute per token, or 4 for accuracy-focused tasks such as coding agents. Tokens are the units in which a model processes text.

2. More loops do not always improve accuracy

Accuracy was measured on the 16-bit baseline. NVIDIA says it used the same evaluation framework built with NeMo Gym and Nemo Evaluator SDK.

SWE-bench Verified, a software issue resolution benchmark, scored 50.3, 59.5, and 61.8 with 1, 2, and 4 loops, respectively. The larger gain from 1 to 2 loops supports the default recommendation.

Terminal-Bench 2.1, a terminal task benchmark, used the same Terminus-2 evaluation framework for comparison models. Scores were 12.5, 18.1, and 22.2 with 1, 2, and 4 loops, respectively.

By contrast, GPQA Diamond, a reasoning benchmark without tools, scored 69.19, 72.41, and 72.16 at the same loop settings. More loops represent a task-specific compute budget, not a guarantee of higher accuracy.

3. Cache only the last loop to prevent memory growth

The KV cache stores information from previous tokens to reuse past computations during generation. The default keeps separate storage per loop. Last-loop caching overwrites one cache per recurrent layer, keeping its size at 1 loop even as loops increase.

This leaves memory available for long inputs or concurrent requests, but does not eliminate repeated computation. More loops also tend to reduce output token counts, so overall latency and cost must be measured on actual tasks.

NVIDIA says its experiments found that last-loop caching delivered accuracy close to the default method and nearly identical output lengths. It did not provide specific figures for the accuracy difference between the two methods.

4. Running it requires source code with support

According to the source, vLLM support is under review in pull request 60149 and requires installation from that source until an official release includes it. llama.cpp also instructs users to build supporting source code before an official release.

The following server launch example is reproduced unchanged from the source. It uses 2 loops and stores a separate cache for each loop.

vllm serve --model nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 \
    --hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "per_loop"}' \
    --max-model-len 262144 \
    --max-num-seqs 128 \
    --enable-prefix-caching \
    --async-scheduling \
    --reasoning-parser nemotron_v3 \
    --tool-call-parser qwen3_coder \
    --enable-auto-tool-choice

To cache only the last loop, change recurrent_kv_cache_mode above to last_loop. The validated context length is up to 256K tokens. A lower maximum is recommended when memory is limited or more concurrent requests are needed.

5. Distinguish deployment formats from measurement conditions

BF16 is the baseline, with both weights and activations at 16 bits. MXFP8 reduces weights to 8 bits while keeping activations at 16 bits.

NVFP4 uses 4-bit weights and 16-bit activations. Pretraining also began in 4-bit floating point before continuing at 16 bits.

On NVIDIA's DGX Spark at batch size 1, with 2560 input and 300 output tokens, 8-bit and 4-bit weights ran inference 1.69 and 2.17 times as fast as BF16. With 128 input and 1024 output tokens, the factors were 1.84 and 2.44. Do not directly compare the two input/output conditions.

The GGUF release contains 5-bit-family quantized files that llama.cpp reads directly. The Apple silicon Mac example sets context to 4096 tokens. The earlier accuracy scores do not apply to this quantized version; its accuracy differences were not provided.

The model's value lies in adjusting compute budgets at a small parameter count, while adoption depends on real-task evaluation with matched loop counts, cache methods, and deployment formats.