devotd.

LLMs

Recent developments, checked against their sources.

Latest stories

57 stories
  1. LLMs KST

    QwenGyre arXiv v1: 59 hours saved in 2.4T training, a 16-hour Colocate gap

    QwenGyre moves GPU capacity between agent rollouts and reinforcement-learning updates as work finishes. In a 48-step NL2RepoBench run with a 2.4T model, it finished 59.13 hours (43.9%) before Async and 16.05 hours (17.5%) before Colocate. The much-quoted 1.85× result belongs to a different 122B TerminalBench test; the paper provides no code link or repeated-run variability.

  2. LLMs KST

    TRL v1.15.0: fused LM head on by default, no full logits tensor

    TRL v1.15.0 scores tokens in SFT, DPO, KTO, GRPO, RLOO and Distillation without building the full logits tensor. In the project's measurements DPO's maximum trainable length grew 5.80×, but the effect depends on the previous loss path and vocabulary size.