Granite Switch 4.1: Unified chat, decision-making and speech recognition, open weights

1. Bundling capabilities with the base model

The model cards list October 14, 2026 as the release date, and the introductory blog appeared the next day. The base model comes in 8 billion and 30 billion parameter configurations. The weights are released under Apache-2.0.

Both configurations include two adapters, small sets of additional weights that add capabilities. The general decision adapter takes a state and choices and returns probabilities for each choice. The safety adapter detects harmfulness, jailbreak attempts and similar risks. A 470 million parameter speech recognizer is also included.

The adapters reuse the cache, which stores computation results from the base model’s processing of the prefix.

The recognizer first converts speech into text, which is then added to the conversation context. This differs from a language model understanding sound directly.

2. A shared server still requires different response handling

For vLLM, install and run the project’s plugin. Below is the server launch example from the 8B model card, used after downloading the repository and entering its directory.

pip install -e ".[vllm]"

python -m vllm.entrypoints.openai.api_server \
  --model ibm-granite/granite-switch-4.1-8b \
  --port 8000

Passing an adapter name in adapter_name makes the chat template insert switching control tokens. General decision calls send the state and question as two user messages, without a system prompt. The design keeps the state identical to regular chat so prefix computation can be shared.

Decision results are read by requesting log probabilities for choice labels rather than answer text. Specify single-token labels and normalize the returned values into probabilities over all choices. The safety adapter instead returns yes/no judgments in JSON.

3. Read decision scores within the evaluation’s scope

The model cards evaluate 1,068 items from the public fast-evaluation subset of Decision Index using v0.3 scoring.

Evaluation ran on hosted vLLM. It used a fixed decision system prompt, a state and choices with single-token labels. Its prompt conditions therefore differ from the general call example described earlier.

On a 100-point scale, the decision adapters scored 42.9 overall for 8B and 50.1 for 30B. Tool-domain scores, shown multiplied by 100, were 59.8 and 76.5, respectively, while both scored 49.8 in retrieval. The larger model’s advantage therefore varies by domain.

In domains not covered by training, general decision probabilities should be treated as a starting point for further judgment rather than definitive classifications.

The cards report that local and hosted evaluations of the same configuration agreed within about 0.6 points. However, hardware and quantization conditions were not provided. These scores alone cannot establish deployment speed or memory usage.

4. Server consolidation trades infrastructure savings for request-handling costs

IBM says the combined model uses roughly as much GPU memory as the base model alone. This reduces the burden of separate servers for each function. Applications must still distinguish probability calculations from safety judgment interpretation by response type.

More choices mean more requests. vLLM limits label log-probability requests to 128 tokens at a time. The cards advise keeping the question unchanged, splitting labels into groups of 128 or fewer, and combining results to calculate probabilities. This is mathematically equivalent to an uncapped call but requires multiple requests.

5. Check speech errors and safety validation separately

Speech input supports English only, with lowercase output and no punctuation. The speech recognizer and language model were not jointly trained, and there is no learned connection between them. Transcription errors can alter subsequent answers or decisions.

The built-in safety adapter does not replace application-level safety testing. The released configuration has no adapters for factuality detection and correction or policy guardrails.

The value of adopting Granite Switch 4.1 depends less on server consolidation itself than on whether function-specific response handling and validation remain manageable afterward.