devotd.
Recent developments, checked against their sources.
Tencent has released a draft model that generates answer candidates in parallel, along with inference code. Tests showed longer accepted drafts than the comparison methods. Actual speed gains must be weighed against extra resource costs.
NVIDIA has released a 24GB vision-language model designed to handle screen understanding and tool calls on personal devices. It compresses a large model and corrects quantization errors through training. Bolt outscored Qwen3.6 on terminal tasks but lagged on some knowledge accuracy and hallucination suppression measures. In separate low-precision speed tests, it was slower than two smaller mixture-of-experts models.
A new path keeps large language model expert weights compressed on the CPU while training only small correction weights. Saving and inference integration are supported, but full-model quality and throughput results are not provided.
Intel has updated the engine behind its local language model inference package and expanded model support and low-precision execution paths. It also offers faster response generation and customized model serving. However, users must check model-specific requirements and note that the image is for demos.
Support now lets users convert and run a German-English open-weight model with a local inference tool. Only some parameters are used per token, but memory for all weights and validation in real-world use remain separate considerations.
Hermes Agent 0.22.0 manages local-model setup, serving and updates, while source installations now follow stable releases by default. Hosted app connectors still use the Nous gateway, so local inference does not mean the entire workflow runs offline.
XeStrata xe0.1.43.1 fixes a regression in xe0.1.43 where Qwen3.8-Flash-Next prefill on Intel GPUs fell back to the DP4a path. xe0.1.42, a day earlier, fixed a separate defect in which GPU machine code inside the packages was swapped.
A Project Maya contributor reported 26.0 tok/s decode on one RTX 3090 and 53.0 tok/s on two. The two-card setup also changed the layer split, MTP drafting and prefill chunk size, so it cannot be read as a scaling factor from GPU count alone.
llama.cpp merged runtime support for Prism Bonsai 2 27B on October 11. The model card and backend docs checked on October 13 still require the Prism fork, so file packing, runtime revision and backend combination need separate checks.
PXA 3.1 adds an Overdrive preset for older NVIDIA cards such as the Tesla P100 and V100, but its speed work assumes PXA quant packs and the PXQN library bundled in the package. Keeping a learned expert map outside a container also needs a named volume.