Ollama v0.40.7-rc0: Reasoning and draft handling improvements, decider model support

1. Handling reasoning settings in Responses

Clients using the Responses API may send the previous reasoning.effort value in their next request even after switching to a model without reasoning support. This setting controls how much reasoning occurs before an answer, and its carryover caused requests to be rejected. With this change, if think is not explicitly set, leftover reasoning settings are ignored and requests also work on models without reasoning support. PR #19010

		if req.Think == nil {
			c.Set("relax_thinking", true)
		}

Relaxed handling applies when the user has not explicitly set think. Explicit values retain their priority and validation, and invalid reasoning level names still cause errors. For requests that require reasoning, it is better to explicitly enable think to check support. PR #19010

Internal follow-up requests after web searches also reuse the previously parsed settings. This also fixes cases where a search succeeded but the next answer failed with an unsupported-reasoning error. PR #19010

2. Automatic draft model type detection where possible

The execution server now automatically detects most types of draft models, which are auxiliary models used to accelerate inference. For ordinary draft models, Ollama passes the file path and predicted token count, leaving type selection to llama-server. This changes the roles of the two components rather than what users must explicitly configure. PR #19064

MTP auxiliary files and Eagle3 are exceptions to automatic detection, so their types are explicitly passed. For models split across multiple files, detection markers in later files are also checked. PR #19064

The existing behavior remains: if the predicted token count is 0 or less, draft model arguments are omitted, disabling inference acceleration. PR #19064

3. PPLX Decider v1.2 support and prerequisites

MLX, the model execution foundation, now supports PPLX Decider v1.2. This decider model scores choices for questions and also handles image input. PR #19072

The model uses non-causal attention, referring to later as well as earlier input. When questions or state change, earlier computations cannot simply be reused, so the implementation does not reuse cached prefix computations. It trades cache reuse for the ability to consider later information. PR #19072

The source notes that the model is in the library, but the manifest list grouping deployment targets has not yet been created, and assumes a minimum version of 0.40.7.

Importing requires readout.safetensors, a separate decider weights file, and a configuration file for combining the decider and questions. The decider weights must retain the BF16 numerical precision format. PR #19072

To run the model, the Modelfile, a configuration file containing model execution instructions, must include the publisher’s system prompt. Requests are rejected if it is missing. PR #19072

Each request contains 1~64 questions, and the question list is split if it exceeds the context limit, the input length that can be processed at once. If even a single question plus the shared input exceeds the limit, an error is returned. PR #19072

4. Balancing convenience and operational overhead

The benefits are fewer conversations blocked by carried-over settings after model switches and draft model type selection aligned with the execution server’s detection capabilities. However, when splitting long requests, the decider model repeats the shared state and image input in each part, which may increase input processing volume. PR #19072 No measurements comparing speed gains or resource usage were provided.

This prerelease focuses on improving request compatibility and expanding the range of supported models.