Strata, a local language model runner, adds image input for a supported model and up to 8 concurrent requests. Developers connecting users or agents gain options and faster generation, but benefits depend on available GPU memory.
1. Selecting image input during setup
Released on October 17, 2026, this version adds image input for Qwen3.6-35B-A3B. Enabling images during setup downloads a 0.9 GB file that connects images to the model and updates the configuration.
The team compared it with llama.cpp using identical images and input tokens. With reference answers fixed, next-token checks matched the first token in 5 of 6 cases. The two-image case diverged at near-tied scores; test-image answers matched.
Enabling images on NVIDIA cards with less than 24 GB now selects the CPU encoder. Converting each image to model input takes 4–6 seconds and uses up to 300 tokens without occupying GPU memory. Existing settings remain unchanged.
2. Processing up to 8 conversations together
Set concurrency with --parallel N during setup or "parallel": N in the config file. A slot holds an active conversation. Setup suggests a count based on memory and context length; pressing Enter keeps one-at-a-time processing.
Batching one token per conversation reads shared model and expert weights once. Experts are selectively used model computations. Tests with 2·4·8 slots, slot reuse and concurrent long inputs produced the same tokens as solo runs, the team said.
Developer measurements. An R9700 32 GB with every expert in VRAM generated 128 new tokens per request, in five interleaved pairs against llama.cpp’s llama-server -np N -fa on. Speedups ranged from 1.14x to 1.61x, with Strata faster in all five pairs at every request count.
| Concurrent requests | Strata | llama.cpp (llama-server) |
|---|---|---|
| 1 | 83.9 | 73.8 |
| 2 | 173.5 | 107 |
| 4 | 252 | 165 |
| 8 | 325 | 208 |
3. Concurrency comes at a memory cost
If all expert weights fit in GPU memory, setup suggests up to 3–4 slots. Otherwise, conversation storage shrinks the expert cache, which keeps weights nearby. Weigh concurrent-request gains against slower individual requests.
On an RTX 3060 12 GB with 8K context and 4 requests, concurrency was 1.15 times as fast as sequential processing with draft-layer token prediction. A single request with those slots was 4% slower. More concurrency does not speed up a solo chat.
4. Faster response generation and input processing
Decoding generates response tokens in sequence. In 18,360 comparisons using actual internal values, the new kernel had no bitwise mismatches with the old one. Default input batches also grew to 4096·2048·1024 tokens, depending on memory.
On an R9700 32 GB with UD-Q4_K_XL, generation rose from 83.2 to 117.8 tokens per second, a 1.42-fold gain across all five alternating test pairs. Processing 4K input rose from 3,453 to 6,220 tokens per second, a 1.80-fold gain.
By contrast, CPU-bottlenecked hybrid execution on an RTX 3060 12 GB improved generation by only 2% and input processing by 3%. The gains on the other card are not a reliable expectation for different hardware.
Prompts shorter than the input batch are unaffected, but longer prompts may produce slightly different outputs after the batch size change. The source provides this option to restore the previous 512-token size.
--prefill 5125. Distinctions to make before deployment
Multi-GPU servers also gain more than 8 slots and optimizations for copying and storing intermediate states. These server features require explicit activation and leave defaults unchanged. They are separate from support for up to 8 requests.
This model was not validated on physical Windows hardware, Intel Arc or RTX 5070. A model recognition issue on Windows AMD was fixed, but that does not mean validation in that environment is complete.
Run UPDATE.bat on Windows or ./update.sh on Linux to update; setup replaces the engine.
This release matters less for its maximum concurrency than for letting users choose image input and throughput gains to suit their hardware's memory and actual conversation count.