Ollama said on September 29 that it supports decision models as of 0.35, and ggml-org announced support in the llama.cpp server on October 2. Both take a state and named questions at the /v1/systemone endpoint.
A decision model answers by scoring the options it is given instead of generating text. Its answer is always one of the options, with a probability attached.
There are three question types. choice returns a probability per option, score an expected level across 2 to 10 levels, and noul the probability of yes for a yes/no question.
In the llama.cpp list, only some models such as OpenJev could read images at the time of writing. Image input is not a common feature of decision models.
Caveats: licenses, per-model confidence and vendor figures
The interface is shared, but licenses differ. OpenJev is CC BY-NC 4.0, while other entries such as Julia-1 and Kev-4B are Apache 2.0.
The llama.cpp post says to pick the confidence cutoff per model. One vague ticket scored 0.25 with Julia-1 and 0.80 with Kev-4B.
Option descriptions also changed results. With bare labels, Julia-1 routed a double-charge ticket to shipping, and to billing (0.99) once descriptions were added.
Speed figures are vendor measurements. llama.cpp gives medians on one RTX PRO 6000, and Ollama says Nimble 9B averaged 91 ms per decision on an M5 Max. This article did not run the models or measure latency.
Labels and a deferral path before routing
For classification or routing, label definitions and a path for deferring judgment come before thresholds. Evaluation inputs include ambiguous and out-of-scope cases, not just typical ones.
A probability concentrated on one option does not confirm accuracy or calibration. Important decisions need a fallback path for when a prediction is hard to trust.
Decision models remove parsing, but which probability to trust is still something to set for each model directly.