Researchers at the Institute of Science and Technology Austria released a compressed large AI model for coding and tool use. They did so by removing some computation blocks and lowering storage precision. Developers running coding models locally get an option with transformer weights reduced to 72.1GB, easing file and memory demands. A transformer is a neural network architecture that learns relationships between inputs; weights are the numerical values of its learned connections.
1. Removing half the expert blocks and storing the remaining values more compactly
The base model is a mixture-of-experts model with 763.2 billion parameters. It selects only some specialized task blocks, called experts, for each input, activating 6 of the 384 experts in each layer per token.
The Coder model removes 192 experts per layer. GSQ quantization lowers the storage precision of the remaining weights, while pruning removes entire expert blocks. The remaining weights retain the IQ2_XXS build's quantization types for each tensor, a multidimensional numerical array. The model card lists that build's rate as 2.00 bits per weight.
2. The 1-bit label is calculated using the original transformer size
The model card's 1.05 bits per parameter is an effective rate: 72.12GB of transformer weights converted to bits and divided by the original transformer's 551.9 billion parameters. This includes removed experts in the denominator, counting them as 0 bits. For the remaining 280.1 billion parameters alone, the average is 2.06 bits.
The separate Engram lookup table was excluded from pruning and the quantization search, and stored at 4.50 bits per weight. Including it, the full model's rate is 1.92 bits per parameter. The files to download are as follows.
Table: Coder model transformer and lookup file sizes
Conditions: GSQ-RCO Coder, half the experts removed, transformer retains IQ2_XXS tensor types, Engram uses IQ4_NL
Unit: GB (10 to the power of 9 bytes)
| Item | Size |
|---|---|
| Transformer shards, storage split across multiple files | 72.12 |
| Engram lookup file | 110.60 |
| Total | 182.72 |
Engram is a lookup file read one row per token and can remain on disk through memory mapping during execution. If you already downloaded the build before pruning, you can reuse the same Engram file.
3. Larger score drops on multistep tasks than on single-pass code generation
According to the researchers, the Coder model achieved the following scores at reasoning effort 50. The results compare the base model and compressed version on each evaluation.
Table: Base model and compressed version scores by coding evaluation
Conditions: reasoning effort 50, base model FP8 / FP4, Coder model with half the experts removed and IQ2_XXS quantization, transformer 72.1GB
Unit: score, base score retention (%)
| Item | Base model | Coder model | Retention |
|---|---|---|---|
| SWE-bench Verified | 81.20 | 77.00 | 94.8 |
| LiveCodeBench v6 | 89.71 | 88.57 | 98.7 |
The card notes that retention on the first evaluation, which involves multiple steps in real repositories, was lower than on single-pass code generation. It interprets this as consistent with errors accumulating across successive steps.
The selection method, RCO, chooses experts to minimize output differences from a quantized reference model. It retains the same number per layer for GGUF compatibility, jointly reducing whole-model output differences rather than ranking experts by individual importance.
4. Focusing on specific capabilities in exchange for storage savings
The transformer shrank from 138.2GB in the pre-pruning build to 72.1GB, while the files including Engram fell from 248.84GB to 182.72GB. Removing experts reduces both storage and resident memory.
For this experimental release focused on code, tool use, long contexts, and image processing, the researchers accepted losses in other capabilities as a compression trade-off. They recommend the unpruned build for general use. Capabilities not represented in the calibration data used to adjust output errors may weaken.
5. Download the files, link the lookup file, and run
Running the model requires llama.cpp b11870 or later. The original commands below download the Coder shards and Engram file, link the lookup file as the sixth shard, and start execution from the first shard.
# download the Coder and the Engram file (requires: pip install -U "huggingface_hub[cli]")
hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF \
--include "IQ2_XXS/*" "*Engram*" --local-dir .
# the Engram file is shard 6 of the Coder
ln -s ../DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf \
IQ2_XXS/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-00006-of-00006.gguf
llama-cli -m IQ2_XXS/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-00001-of-00006.gguf -ngl 99This model's practicality should be judged less by its small transformer size than by how well its reduced capacity is focused on coding tasks.