The Deep Algorithms and Systems Lab at the Institute of Science and Technology Austria released weight files that compress a 763.2 billion-parameter model using different precision levels across its parts. According to the lab, a dedicated runtime enables inference on a single NVIDIA GPU with 12GB of memory. This expands options for developers testing large models on their own hardware.
1. Weight files released in three sizes
The release includes three builds of weight files in standard GGUF format. Each build splits the model's backbone weights across four files and provides a separate Engram lookup table file. This shared file is identical across builds and only needs to be downloaded once.
Table: File sizes by quantized build
Conditions: GSQ-RCO builds of the same model; backbone size is the sum of four shards, and the Engram file is shared by all three builds
Unit: Sizes in GB (1 billion bytes); bpw is the average bits per backbone weight
| Build | bpw | Backbone | Engram | Total |
|---|---|---|---|---|
| IQ1_M | 1.75 | 120.72 | 110.60 | 231.32 |
| IQ2_XXS | 2.00 | 138.24 | 110.60 | 248.84 |
| IQ2_XS | 2.25 | 155.23 | 110.60 | 265.83 |
The backbone contains 551.9 billion parameters, and the Engram file contains 196.6 billion parameters across two lookup tables. The source recommends IQ2_XXS as the build closest to the original. Even the smallest configuration totals 231.32GB.
2. More precision for sensitive weights
Non-uniform quantization varies compression strength for each tensor, a group of weights. Rather than applying the same quantization type to every tensor, it allocates more precision to quality-sensitive parts while meeting the overall file size target.
GSQ quantizes individual tensors to low bit widths, while RCO searches for the type to apply to each tensor within a fixed size budget. Expert weight types are assigned at the layer level to fit the GGUF format.
3. A 2-bit build that retains the original scores
The model card's benchmarks were measured at reasoning effort 50. The baseline is the original release with FP8 dense weights and FP4 routed experts. The three-task averages are also reproduced exactly as listed on the card.
Table: Benchmarks for the original model and quantized builds
Conditions: Reasoning effort 50; baseline uses FP8 dense weights and FP4 routed experts
Unit: Scores; bpw is the average bits per backbone weight; recovery is the ratio to the baseline model's task average (%)
| Model | bpw | AIME25 | GPQA-Diamond | LiveCodeBench v6 | Task average | Recovery (%) |
|---|---|---|---|---|---|---|
| Original | — | 100.00 | 90.91 | 89.71 | 93.54 | 100.0 |
| IQ1_M | 1.75 | 100.00 | 88.38 | 88.57 | 92.32 | 98.7 |
| IQ2_XXS | 2.00 | 100.00 | 90.91 | 89.71 | 93.54 | 100.0 |
According to the lab, IQ2_XXS matched the original on all three evaluations, while IQ1_M averaged 92.32 points, or 98.7% of the baseline.
4. Benefits and costs of local execution
The card says Strata runs all three builds on a single 12GB NVIDIA GPU and provides a one-click installer and an OpenAI-compatible endpoint on localhost, an address pointing to the same computer.
Engram reads one row per token, enabling disk memory mapping, which treats files like memory. The backbone is not loaded entirely onto the GPU; the remainder stays in system memory. The card advises storing the Engram file on an SSD, a solid-state storage device.
This setup lowers GPU memory requirements by shifting the burden to large storage capacity and memory outside the GPU. The smaller IQ1_M has a lower total file size than IQ2_XXS, at the cost of a lower average score. The 2-bit IQ2_XXS is larger but retained the original scores on the three published evaluations.
According to the card, llama.cpp does not support this model as of October 23, 2026. The command below is an example for when support becomes available; current execution uses Strata. The card lists IQ2_XS benchmark results as pending measurement.
llama-server -m DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS-00001-of-00004.gguf -ngl 99 \
--reasoning on --reasoning-effort 50This release offers compressed builds that retain the original evaluation scores through per-tensor precision allocation, while Strata enables local execution by shifting GPU memory demands to system memory and storage.