devotd.
Recent developments, checked against their sources.
Hermes Agent 0.22.0 manages local-model setup, serving and updates, while source installations now follow stable releases by default. Hosted app connectors still use the Nous gateway, so local inference does not mean the entire workflow runs offline.
XeStrata xe0.1.43.1 fixes a regression in xe0.1.43 where Qwen3.8-Flash-Next prefill on Intel GPUs fell back to the DP4a path. xe0.1.42, a day earlier, fixed a separate defect in which GPU machine code inside the packages was swapped.
A Project Maya contributor reported 26.0 tok/s decode on one RTX 3090 and 53.0 tok/s on two. The two-card setup also changed the layer split, MTP drafting and prefill chunk size, so it cannot be read as a scaling factor from GPU count alone.
llama.cpp merged runtime support for Prism Bonsai 2 27B on October 11. The model card and backend docs checked on October 13 still require the Prism fork, so file packing, runtime revision and backend combination need separate checks.
PXA 3.1 adds an Overdrive preset for older NVIDIA cards such as the Tesla P100 and V100, but its speed work assumes PXA quant packs and the PXQN library bundled in the package. Keeping a learned expert map outside a container also needs a named volume.
Basalt forks Strata into an engine for one target, Qwen3.8-Flash-Next on Linux and NVIDIA Blackwell (sm_120). The developer’s reported 665 and 354 tok/s come from a particular model pack, workload and tuned configuration; built-in defaults and context splitting under concurrency shape the result just as much.
strata-mlx is an independent MLX engine that reads Strata's model files. A user's measurements on a real 32GB MacBook Air showed that cutting the expert budget from 20.5GB to 16GB nearly doubled generation speed and sharply reduced cumulative swap-out.
audio.cpp v0.9.1 replaces the PocketTTS CPU Mimi upsampler’s 32 MiB dense diagonal weights with a 64 KiB depthwise module. In the maintainer’s measurements, four-thread decoder time per frame fell from 30.50 to 10.11 ms, but GPU paths are unchanged and some release binaries may require AVX-512.
On October 8 MSI opened Hong Kong preorders for the Prestige N16 Flip AI+, an RTX Spark-based 2-in-1. Up to 128GB is shared by the CPU and GPU, and for local AI the question is whether a real software stack on Windows on Arm runs within that space.
WHIRL v0.1.4 narrows the CORS default of its native Windows LLM server to web pages served from loopback addresses. Inference code and speed are the same as v0.1.3, and a streaming client that stops reading no longer blocks other users.