devotd.
Recent developments, checked against their sources.
A project for running large AI models on personal PCs has changed its name and consolidated several model engines into one repository. The result of about 27 tokens per second on one GPU required ample RAM and fast storage. It does not imply quality parity with official services.
Support for matrix multiplication with ultra-low-bit weights, row extraction, and Hadamard operations has been added to Snapdragon NPUs. A developer said the Bonsai-2 1-bit and 2-bit paths are usable, but further optimization is needed to better exploit the reduced memory footprint.
Cloudflare introduced a system that continually evaluates verified bots' compliance with instructions and responses to requests to slow down. It is building an internal observation history, with public scores and customer-selected trust criteria planned for the future.
Users can now search Hugging Face Hub models in the web interface and download inference auxiliary files matched to a quantization. Paused downloads survive restarts, but start over if files change or resume requests are rejected.
NVIDIA has released a 4-billion-parameter language model that adjusts accuracy and compute by changing internal loop counts without retraining. It offers last-loop caching and multiple precision formats, but latency and cost need task-specific evaluation.
A security breach affected 1.4% of all console users, specifically some users who connected external AI services. Notified users should revoke and replace keys in provider dashboards and review recent usage and connected applications.
Claude Code added settings to select the auto mode decision model and toggle idle conversation compaction. Alongside the previous version's unattended retry wait cap, this version rejects image sources larger than 128 MB.
The local language model runner adds image input and up to 8 concurrent requests while improving response generation speed. With limited memory, concurrency can slow a single request, so settings must suit the hardware and usage.
Code can now read annotations supplied by external tools and format execution errors from eligible functions through a shared handler. Automatic history compaction is deferred while approval is pending. Existing error handlers need checks for the new error kind.
Liquid AI has released a model for on-device agents that reasons before answering while using only part of its parameters for computation. It showed strengths in tool-use evaluations. Knowledge and coding limitations, weight storage demands, and commercial-use terms also warrant review.