TRL v1.15.0 was released on October 8. SFT, DPO, KTO, GRPO, RLOO and Distillation now score tokens with a fused LM head, and this path is on by default.
A Triton kernel projects hidden states through the LM head in tiles and reduces directly to per-token log-probs and entropy. So the [batch, seq, vocab] logits tensor is never built.
Scoring needs Triton on a GPU, and the release notes name Linux with CUDA, ROCm or XPU.
How the logits tensor takes up memory
The full logits tensor grows with the number of scored token positions and the vocabulary size, independently of model size. With large vocabularies and long sequences, this intermediate tensor easily becomes the memory bottleneck.
If training only needs reduced per-token statistics, the tensor can be skipped. That leaves room for longer examples without shrinking the model.
Fitting longer examples is, however, a different matter from processing the same data faster.
Caveats: the conditions of the project's measurements
The release notes report DPO's maximum trainable length rising from 10,240 to 59,392, a 5.80× gain. KTO gained 6.89×, and scoring-only GRPO and RLOO gained 4.00× and 4.26×.
The runs used gemma-3-1b with random weights, a 262k vocabulary, bf16, batch size 1, gradient checkpointing and sdpa, with a B300's allocator capped at 79GiB to stand in for an H100 80GB. Real H100 hardware, flash-attention, real generation and multi-GPU were not measured.
At 8,192 tokens peak memory fell 52–82%, but steps were only 2.3–10.9% faster. The default SFT loss (chunked_nll) already avoided full logits, so its maximum length was unchanged.
How it differs from other memory-saving techniques
According to the version-pinned TRL memory guide, gradient checkpointing skips storing intermediate activations and recomputes them in the backward pass, and it is on by default in all trainers.
Activation offloading moves activations to CPU RAM to lower peak memory at the cost of slightly longer training. Truncation discards tokens beyond the length limit, so they do not contribute to training.
Each of these pays a different cost, while the fused LM head avoids building an output tensor that is not needed.
Settings that break on upgrade
The full-logits scoring path is gone from GRPO, RLOO, DPO and KTO. A PEFT adapter on lm_head now raises an error; modules_to_save=["lm_head"] is the replacement.
In the same four trainers use_liger_kernel=True is deprecated and will be removed in v2.0.0. DPO and KTO's compute_metrics and compute_loss(..., return_outputs=True) still receive full logits through an extra forward pass, but that path is also slated for removal in v2.0.0.
An evaluation that starts from the current loss path
If the existing setup already avoided full logits, this change may not remove any bottleneck. If it did build the full tensor, comparing peak memory and step time on the same examples and batch settings shows the difference.
Changing context length and other optimizations at the same time makes the cause of any regression hard to pin down.
TRL v1.15.0's fused LM head pushes the memory limit far out for setups that built full logits, but setups that already avoided that tensor will not see the same multiple.