Microsoft Research Asia unveiled Agent Lightning v1.0, connecting production agents to reinforcement learning without rebuilding them. Routing agents' model requests through the framework's proxy preserves their existing execution structure. Developers can thus avoid creating separate replicas of agents for training.
1. Using the real harness instead of a training replica
A harness is the execution framework that coordinates an agent's context management and tool use. In earlier agent reinforcement learning systems, the training framework managed interactions with the environment. This required reimplementing the harness inside the training environment.
Agent Lightning places a language model proxy between the agent and the model. Developers redirect existing model calls to the proxy, and the agent runs as usual. The framework records calls and responses for training. v1.0 calls this approach 'Harnessed Agentic RL,' in which production agent harnesses participate directly in reinforcement learning.
2. Three components connect agent execution and training
According to the company, the entire framework, a foundation integrating features to aid development, is about 3,500 lines long. It consists of an API gateway that records model calls, a controller that runs agents, and a trainer that turns execution results into training data. The controller runs agents as local processes or standard Kubernetes jobs. The verl-based trainer uses the results to build training samples.
Developers connect through the existing harness's model endpoint. Redirecting it to the gateway generally enables training integration. Separating execution from learning reduces the work needed to move the harness itself into the training framework.
3. Treating harness-generated samples as training units
When the harness manages execution, model requests and responses arrive at the training system in sequence. Each task produces multiple samples. Grouping them by task is essential to prevent tasks with more samples from dominating learning.
The researchers applied task-level advantage computation, which measures how much better an action was than a baseline, and loss normalization, which balances the weighting of training errors. The company presented a graph covering 200 training steps on the SWE-smith validation set with Qwen3.5-9B. Using both methods yielded a validation reward of about 0.37, the highest among the compared approaches.
With only advantage computation changed to the task level, policy entropy, a measure of uncertainty in response selection, rose to about 0.65. The company said using both methods kept this value lower and more stable.
4. Alternating execution and updates on the same GPUs
When agent runtimes vary widely, synchronous training can leave GPUs idle while waiting for the slowest run. Fully asynchronous approaches improve utilization but allocate separate GPU resources for execution and training. Agent Lightning v1.0's 'Collocated Async RL' alternates agent execution and model updates on one GPU pool.
Once enough execution results accumulate, new requests pause briefly, in-flight requests finish, and the model is updated. Execution then resumes, so the external harness does not need to manage this transition itself.
The company reported roughly 2 times the overall throughput of synchronous reinforcement learning while using fewer GPUs than conventional asynchronous approaches. The original resource allocation diagram shows an example with 4 GPUs each for synchronous and collocated approaches, and 8 GPUs for the asynchronous approach.
5. Weighing performance gains against operational demands
Using the public SWE-smith dataset and the mini-SWE-agent coding agent, the company built a pipeline linking multiple processing stages. It covers data cleaning, environment setup, defenses against reward hacking, and reinforcement learning.
According to the company, applying only reinforcement learning to Qwen3.5-9B with about 6,000 samples raised its first-attempt problem-solving rate on SWE-bench Verified from 41.8% to 56.4%. The original report gives the increase as 14.6 percentage points.
Support for standard Kubernetes jobs lets teams use their own clusters, which pool multiple computers, as well as cloud or local infrastructure. This can reduce reliance on commercial sandbox services that isolate code for testing. Running multiple agents concurrently still requires computing resources such as CPU and memory. Teams choosing to operate their own infrastructure must manage it themselves.
Agent Lightning v1.0 narrows the gap between training and deployment by connecting real harnesses instead of training replicas, while leaving teams responsible for operating execution infrastructure.