An open-weight language model focused on German and English can now run with a local inference tool. Developers hosting it themselves gain a new execution path, but must distinguish the model size used for computation from the size held in memory.

1. Scope of the new support

llama.cpp b11709, published on October 17, 2026, is a prerelease. Its release notes cover weight conversion, inference implementation, and architecture test registration for the new model. This adds execution support, not just a model name to a list.

The supported model is Aleph Alpha's Kolibri-1. Its model card lists October 3, 2026 as the model release date, distinct from this inference tool's support date. The card's Apache 2.0 license applies to the repository's published weights and configuration files.

2. Trading less computation for more memory

The model uses a mixture-of-experts architecture that selects some of several experts to share computation. It has about 78 billion parameters in total, but activates about 3.46 billion per token, a unit of text processing. The card says each layer selects 6 experts and also has 1 shared expert.

The benefit is that not all parameters are used for every computation. However, the model card says the full model must remain in memory even when only part is active. The published 8-bit floating-point weights occupy about 78 GB. Active parameter count alone does not imply that the model can run with the memory budget of a small model.

3. Changes in the conversion implementation

The implementation alternates four layers attending to nearby text with one attending to the full context. The local range covers the preceding 512 tokens and the current token. The converter reuses an existing mixture-of-experts model class. It records details such as shared expert size and per-layer attention range patterns in the GGUF model file format.

The following conversion code appears in the pull request. It writes model structure information to a file rather than serving as an execution command.

    def set_gguf_parameters(self):
        super().set_gguf_parameters()
        self.gguf_writer.add_expert_shared_feed_forward_length(self.hparams["shared_expert_intermediate_size"])
        self.gguf_writer.add_sliding_window_pattern([t == "sliding_attention" for t in self.hparams["layer_types"]])
        self.gguf_writer.add_expert_gating_func(gguf.ExpertGatingFuncType.SIGMOID)

4. What converted sizes and measured speeds mean

The pull request author reported that converting the original model produces a 156 GiB BF16 GGUF file. A Q4_K_M file using quantization, which represents numbers with fewer bits, is 45 GiB. These are converted file sizes, not necessarily the total memory required during execution.

Performance testing used the Q4_K_M model on a server with four A800 cards. The author reported 2558.18 tokens per second for processing 2048 input tokens and 106.01 tokens per second for generating 512 tokens. Input processing and response generation are different tasks, so these figures should not be read as a single speed.

The sources provide no speed comparisons with other hardware or accuracy changes before and after quantization. These results therefore describe a run on that server, not a guarantee of perceived speed or quality on personal hardware.

5. Limits to check before use

The model card says context, the span of text handled at once, was validated up to 1,048,576 tokens. The base training context is 262,144 tokens. It recommends staying at or below that length for latency- or throughput-sensitive use and complex tasks. Long-context validation is the model maker's claim, not a test of this implementation under identical conditions.

The card's installation and execution instructions are for vLLM. Steps for enabling reasoning mode and tool calling also target that tool. The sources provide neither specific llama.cpp conversion and execution commands nor confirmation of equivalent support for those two features. Adopters must separately verify the model's own capabilities and the support confirmed in their inference tool.

This support adds another option for running the new model; adoption decisions should prioritize memory budgets and validation on actual work over active parameter counts.