Ollama released a candidate that speeds up response generation and long-input processing on Apple silicon while preserving more detail in some small images. Developers running models locally can assess the gains based on whether reading input or generating responses takes more time in their workloads.
1. Drafting tokens first to accelerate response generation
Version 0.40.12-rc0, released on October 27, adds DFlash2 support. It accelerates response generation by drafting multiple tokens before the model verifies them.
The support was added to the MLX execution path, the model computation framework for Apple silicon. It drafts 7 tokens per round and verifies them through the existing state restoration path.
The pull request author measured performance using the qwen3.8:27b model in NVFP4, a 4-bit floating-point numerical format. Both methods ran in the same build and session.
The comparison method, MTP, is a prediction feature built into the model. Below are medians from 16 requests with 2K input on an M5 Max, with the author's reported input and generation throughput shown in separate columns.
Table: Throughput by drafting method on M5 Max
Conditions: M5 Max, qwen3.8:27b (NVFP4), 2K input, median of 16 requests, temperature 1, top_p 0.95, top_k 20, thinking on, same build and session
Unit: tokens per second
| Method | Input throughput | Generation throughput |
|---|---|---|
| Built-in MTP | 767 | 57.0 |
| DFlash2 | 790 | 71.0 |
Under these conditions, generation throughput changed more than input throughput. With 32K input on the same model and device, generation throughput rose from 54.0 to 63.2. That result is the median of 12 requests with all other settings unchanged.
2. Improvements to reading long inputs
Updates to MLX and its binding layer target prefill, when the model first reads the input. They improve throughput for Gemma 4, Flint, and Laguna models using an operation that specifies the input range to examine at once.
In Apple M5 Max measurements shared by the pull request author, the Laguna model's throughput for 2048-token input changed as shown below. The improvement rate is also the value given in the original source.
Table: Changes in Laguna throughput for 2048-token input
Conditions: Apple M5 Max, laguna-xs.2:nvfp4, 2048-token input, prefill throughput before and after MLX and MLX-C updates
| Metric | Before update | After update | Improvement (%) |
|---|---|---|---|
| Throughput (tokens per second) | 4668 | 5795 | 24.1 |
With 6144-token input on the same device and model, throughput rose from 4289 to 5448 tokens per second, a reported gain of 27.0%. This is relevant to input-heavy workloads; decoding speed was unchanged in those measurements.
3. Preserving small-image detail at the cost of more tokens
Gemma 4's compact vision tower, a small component that processes images, was also fixed. Rounding down the smallest grid during automatic resizing could allocate fewer than the intended 70 tokens, losing small-image detail.
The change rounds that grid up without exceeding the next supported budget. When splitting an image into small cells for the model to read, it slightly increases the cell count to reduce detail loss.
Better detail retention comes with the potential cost of using more tokens for image input. Inputs with explicit budgets, larger vision towers, and unified towers retain the existing resizing behavior.
4. Checking the generation path with a test model
The DFlash2 pull request author published a test model under their own name. The download command provided in the original source is:
ollama pull dhiltgen/qwen3.8:27b-dflash2-nvfp4This Ollama release candidate improves response generation, long-input reading, and small-image detail retention, giving users reasons to evaluate it against their workload bottlenecks.