QwenGyre arXiv v1: 59 hours saved in 2.4T training, a 16-hour Colocate gap

Idle GPUs in fixed and shared training pools

Colocate alternates the whole GPU pool between generating agent runs and training. A few slow runs keep the rest of the pool waiting. Async reserves separate pools, but its training GPUs can sit idle while the next batch is prepared.

A separate cost comes from branching agent histories: retries, compression and delegated work share long prefixes. Flattening every branch into an independent sample repeats those shared tokens.

There is also a data bottleneck. Context compression, subagent delegation, and retries turn a run into a branching graph rather than a linear trajectory. Expanding every path into a training sample duplicates shared prefixes and sharply increases training costs.

Moving cells between generation and training

QwenGyre uses Megatron for training and SGLang for rollouts, dividing GPUs into cells. Each cell serves rollouts or hosts one training actor replica. The core cell holds the authoritative weights and the sole optimizer state.

Cells switch to training one at a time when two conditions hold. Remaining rollout capacity must handle unfinished runs. Ready training groups must stay above a set fraction of the batch for a set duration. Satellite cells pull core weights through Mooncake Transfer Engine and RDMA to join the ongoing batch.

During role switching, the proxy routes requests to other engines. It transfers KV cache blocks through RDMA to reduce prefix recomputation. In the 122B NL2RepoBench experiment, average switching times were 8.52 seconds for rollout→training and 3.46 seconds for training→rollout. These times are negligible relative to runs lasting hours.

There is a tradeoff. Streaming training assigns ready groups to updates in order. With multiple steps per burst (μ>1), the entire minibatch cannot be globally shuffled before the burst starts. The authors state that they did not separately measure how sample order affects training quality.

The trajectory processor records model calls as token-in/token-out (TITO) and builds a shared prefix tree. It masks output tokens shared by multiple paths so they enter the loss only once. Evaluable partial work from timed-out runs is also scored. Evaluation failures are distinguished from valid scores of 0 and excluded from training. However, the paper has no ablation isolating partial scoring.

The default per-run trajectory cap J_max is 5. In the 122B NL2RepoBench ablation over steps 1–42, a cap of 5 produced training scores similar to the unlimited setting. It reduced average forward–backward time per step by 25.2%. This is an effect on the training computation phase, not end-to-end time.

Why the 2.4T gain narrows to 1.21×

Comparisons used the same GPU budget and the same target mean policy-version lag (staleness), E[d]=1 or 1.5. The 2.4T experiment trained on NL2RepoBench for 48 steps using 4 cells of 48 nodes each (192 nodes total). The GPU model was not disclosed.

In this experiment, QwenGyre took 75.42 hours, Async 134.55 hours, and Colocate 91.47 hours. The paper reports gains of 1.78× and 1.21×. In time terms, QwenGyre finished 59.13 hours (43.9%) earlier than Async and 16.05 hours (17.5%) earlier than Colocate, the faster baseline.

Values reported in the paper. Time to train 48 NL2RepoBench steps on four 48-node cells (192 nodes in total); speedups are as stated in the paper.

The abstract's "up to 1.85×" comes from different conditions. For the 122B model on TerminalBench over 24 steps with E[d]=1.5, the gain was 1.849× versus Colocate. Under the same conditions, the gain versus Async was 1.430×. Thus, which baseline shows the larger gain reverses across workloads.

The narrower gap versus Colocate at 2.4T relates to the run distribution. Of queries with complete duration records, 61.1% lasted 4 hours or longer (9.51% for 122B). The paper states that the longest run ended at the overall timeout in 38.2% of these queries. Query duration is the duration of the longest rollout run for each query.

The authors interpret completions clustered near the timeout limit as leaving less room to reclaim cells early and gradually expand training. The large gain versus Async is explained by assigning GPUs locked into training by the fixed partition to rollouts. Results are presented without variance across repeated runs, so the variability of this difference is unknown.

The evaluation pass rate in the same 2.4T experiment rose from 52.48% to 58.54% (6.0 percentage points in the abstract). However, the paper itself states that the three schedulers had similar training score curves. This increase is therefore better read as a result of 48-step policy training, not scheduler superiority.

There is no suitable external reference point either. In the original NL2Repo-Bench paper covering 104 tasks, the highest score was 40.2% for Claude-Sonnet-4.5 (Claude Code). Different measurement times, models, and harnesses prevent placing it on the same leaderboard as 58.54%. No separate live leaderboard was found either.

A bounded check with verl step switching

In the 122B, 12-step ablation, adding streaming to Async reduced time by 11.6%. Fine-grained cell assignment was 1.19× faster than a streaming Colocate variant. The whole gain therefore cannot be assigned to GPU reassignment alone.

QwenGyre's code is unavailable, but open source can partly measure the same question. How much time is saved by lending training GPUs to rollouts while they wait for the next batch? In verl V1 Async Trainer, separate_async keeps generation running on independent rollout GPUs. Experimental step switching lets training replicas join the rollout load balancer while waiting for samples for the next step.

trainer.use_v1=True \
trainer.v1.trainer_mode=separate_async \
trainer.v1.separate_async.hybrid_rollout.enable_switch=True

The metrics are the distribution of each query's longest run duration, clustering near the timeout limit, and training GPU idle time. If the distribution clusters near the timeout, resource reallocation appears more likely to contribute less, as in the 2.4T case.

However, verl lends GPUs between steps, while QwenGyre lets cells gradually join an ongoing batch. Results from verl cannot validate QwenGyre's speedup.

Asynchronous RL and partial rollout are already common. slime v0.4.0 collects completed groups into training batches without waiting for all workers. AReaL overlaps rollouts and training on separate GPUs. Within this comparison, QwenGyre appears distinct in combining gradual cell-level joining, request transfer without interrupting the harness, and branching trajectory processing.

What the unpublished code leaves uncertain

Each cell must accommodate model memory and training parallelism, making this setup difficult with a very small GPU budget. The authors also state that they did not evaluate efficiency with a much smaller total GPU budget. Applying the 192-node figures to clusters with tens of GPUs appears difficult.

The checked arXiv v1 has no implementation repository link, and MarkTechPost points this out too. This confirms only the absence of a link and is not independent validation.

The next points to examine are whether code is released and how the gap versus Colocate changes with distributions less clustered near the timeout or with smaller budgets.

The 16.05-hour Colocate advantage in the 2.4T test depends heavily on when long-running agents finish; without code and smaller-scale results, it is a bounded research result rather than a deployable speedup promise.