devotd.
Recent developments, checked against their sources.
AIBrix v0.8.0's experimental ModelClaim separates model lifecycles from Pods and puts idle engines to sleep to share GPU memory. However, the first request to a sleeping model receives HTTP 503 rather than waiting for a result. If its reserved space is also shared while it sleeps, waking up depends on available capacity.
vLLM TPU Inference v0.31.0 adds optional dispatch that reduces inter-chip MoE communication and W4A16 linear operations on the JAX path. Development PRs reported a 4.5% reduction in total decoding step time for one MoE configuration and fit Gemma 4 31B QAT on one v6e chip. These results address different bottlenecks; they do not establish general speedups or preserved quality.
The Ling-3.1-flash model card advertises a 1M-token context, while SGLang's deployment guide specifies 262,144 tokens. Active-parameter figures also differ: 25B versus about 17B. Self-hosting plans need the actual checkpoint and server configuration, not just the model description.
Mistral Large 4 opened its public API preview on October 6, while model weights were promised for month-end. Hugging Face still marked them as upcoming on October 14. Evaluating the preview service and planning future self-hosting require different evidence.
vLLM 0.31.0 keeps processed model weights in a GPU daemon so a restarted engine can avoid loading them again. But its checkpoint fingerprint checks safetensors headers, not weight values; different fine-tuned models can therefore match the same cache entry. One user measured roughly 32 seconds per restart rereading 66 headers on remote storage.
QwenGyre moves GPU capacity between agent rollouts and reinforcement-learning updates as work finishes. In a 48-step NL2RepoBench run with a 2.4T model, it finished 59.13 hours (43.9%) before Async and 16.05 hours (17.5%) before Colocate. The much-quoted 1.85× result belongs to a different 122B TerminalBench test; the paper provides no code link or repeated-run variability.
Parsimony counts how much code the patches from public coding-agent benchmarks add. The website ranks by mean net coding units over measurable solved attempts, with the solve rate and measurement coverage shown in separate columns.
TRL v1.15.0 scores tokens in SFT, DPO, KTO, GRPO, RLOO and Distillation without building the full logits tensor. In the project's measurements DPO's maximum trainable length grew 5.80×, but the effect depends on the previous loss path and vocabulary size.
On October 5 Reflection announced Beam, a 501B-parameter MoE model, and began opening its beta API gradually. For moving an existing agent, what matters is the service token limits, the supported endpoints and parameters that are accepted but have no effect.
Ai2’s AstaBrief 8B is a Qwen3-8B-based model that turns a question and retrieved literature excerpts into a cited report in one pass. Evidence retrieval belongs to the input path, and while the model is Apache 2.0, the prompt and SFT/DPO data are CC BY-NC 4.0.