Strata, for users running AI models on their own computers, has released an update with new model file support and better multi-GPU execution. Users can select compact quantized files in the app. Some card setups can automatically use a new method that distributes model computation.
1. Seven new quantization sets work in the app and terminal
This version supports seven Dynamic v3.1 quantization sets reuploaded by Unsloth on November 4. They target Qwen3.8-Flash-Next. Users can download, prepare, and run them in the app or terminal, or load files downloaded separately.
Download sizes for each set are listed below. Files are verified against the specified revision using their sizes and SHA-256 hashes, which summarize file contents.
Table: Download sizes of newly supported quantization sets
Conditions: Unsloth Qwen3.8-Flash-Next, revision d9620119, all download files for each set
Unit: Download size in GB
| Set | Download size | File count |
|---|---|---|
| UD-IQ1_M | 68.0 | 3 |
| UD-Q2_0 | 70.6 | 3 |
| UD-IQ2_XS | 75.8 | 3 |
| UD-Q2_K_XL | 79.6 | 4 |
| UD-IQ3_XXS | 83.6 | 4 |
| UD-Q3_K_XL | 89.5 | 4 |
| UD-IQ4_XS-v31 | 93.7 | 4 |
The final file in each set is a 28.8GB table the model reads for every token. The new UD-IQ4_XS-v31 differs from UD-IQ4_XS in file layout and tensor types, or multidimensional numeric arrays. It is offered separately, and the existing set remains available.
In the terminal, specify the desired set in place of the model name in this command.
strata launch --family unsloth --model <name>2. Four Radeon cards compute each model layer together
On Linux, automatic selection picks a new splitting engine for IQ3_S variants running on four Radeon AI PRO R9700 cards. Layer splitting assigns different layers to different cards. The new engine distributes each layer's computation across all cards. Two- and three-card setups retain the previous layer splitting.
The developers report 32.7% higher aggregate response generation throughput for 8 concurrent users on the same four cards with UD-Q4_K_XL, versus the 0.2.10 package. The two previous readings were 276.3 and 263.3 tokens per second, versus 358.4 and 357.4 with the new engine. Single-user performance improved by 6.9%. Processing time for two prompts, the inputs sent to the model, of about 103K tokens improved by 8.1%.
The comparison used two paired tests alternating between the existing package and a setup with its engine file replaced, with one startup per run. The new engine raises the default request slots for concurrent processing from 6 to 8 and changes how results are summed across cards.
3. NVIDIA processes input in quantized form
On NVIDIA GPUs with compute capability 8.0 or higher, indicating GPU instruction support, integer kernels for K quantization default to Unsloth UD-Q4/Q5/Q6_K_XL files. These GPU computation programs operate on compressed weights rather than converting the model's learned numeric weights to 16-bit numbers as before.
RTX PRO 6000 measurements showed the input processing gains below. MTP, which predicts upcoming words to aid verification, was enabled. All experts, the model's specialized computation components, were resident on the card.
Table: RTX PRO 6000 prompt processing speed
Conditions: RTX PRO 6000(sm_120), UD-Q4_K_XL, all experts on-card, MTP, context 131072, median of 4 requests per cell, 2 alternating rounds, engine restarted for each path in each round
Unit: Tokens/second
| Prompt length | Previous path | New path |
|---|---|---|
| 4K | 2,338 | 3,097 |
| 8K | 3,389 | 4,241 |
| 20K | 3,386 | 4,573 |
| 32K | 3,709 | 4,886 |
The developers report a 32.6% gain for 32K input. This kernel change did not alter response generation speed. It improves input processing, so users should expect changes in the processing period between sending a long question and the start of the answer.
4. Starting with less GPU memory means slower long inputs
For layer splitting across multiple NVIDIA cards with little spare memory, insufficient memory triggers gradual reductions in on-card weights, auxiliary models, and other allocations until startup succeeds. This does not reduce the user-selected context size, the limit on input the model can reference.
Instead of actual smaller cards, the developers tested two RTX PRO 6000 cards with available memory limited to 8GB each. They used Linux, Qwen3.8 IQ3_S, 128K context, 8-bit storage for attention state recording token relationships, and automatic speculative model selection. The previous version failed to launch in all six attempts. The new method started and processed 32K input in 14.5 seconds in an engine test.
The benefit is being able to start with less GPU memory; the trade-off is longer input processing time. The developers reported 1000–2200 tokens per second for the small-card setup, below 6000–7000 for the large-card setup. The input processing path for smaller cards still uses only one card.
5. Check both input processing and response changes after updating
In the app, use the built-in updater or a new installer. The terminal update command refreshes both the control components and engine, keeping the previous engine for rollback.
These performance figures mainly came from release-source builds and packages with replaced engine files, not fresh measurements of the official engine package. The developers also noted that rounding differences due to kernels and input batch sizes can change the first answer to some agent questions.
Strata 0.2.11 makes new quantized files easier to run and supports more card setups, but input processing gains should be assessed separately from response generation speed.