Sage-A1.5: Scientific agents, orchestration and verification

Evaluation methods

Search evaluations use search, browsing, and code tools. BrowseComp restarts from scratch when tool calls exceed 300 or the context limit is reached, allowing up to 5 rounds. SWE evaluations enable automatic context compression in mini-SWE-agent and block network access and Git history that could reveal answers. MLE-Bench-Lite allows 12 hours per task on a single H200 GPU across 22 Kaggle competitions. The bronze-medal-or-better rate is averaged over 3 runs with different random conditions.

Results

Table: Comparison of key agent and engineering benchmarks

Conditions: Selected representative comparisons with the previous generation from Table 1 of the company's model card.

Unit: Score

SWE-bench Pro evaluates modification tasks in complex repositories. Gains in both metrics indicate improvements in code modification and model training and submission tasks.

Scientific research results

Table: Comparison of scientific research and scientific agent benchmarks

Conditions: Selected scientific research metrics and comparison models from Table 1 of the company's model card.

Unit: Score

FrontierScience-Research covers doctoral-level research subtasks, while CMT covers condensed matter theory problems. It leads on both but trails a large comparison model on CritPt, a physics research evaluation that verifies answers computationally. Research assistance also requires evaluations suited to the field and verification method.

Why the results differ

Sage-A1.5 is a 35B mixture-of-experts model based on Qwen3.6 and Shanghai AI Lab's Agents-A1. The company describes its approach as 'composing capabilities.' Orchestration distributes questions to subagents with independent contexts and combines evidence. Verification revisits evidence and failures through internal reasoning without an external judge.

Section 2.6 of the report describes combining outcome-based reinforcement learning with token-level process guidance from domain-specific teachers after learning from examples. The mode-switching experiments below should not be taken as proving all causes of the overall gains over the previous generation.

Table: Evaluation results by orchestration mode

Conditions: Technical report Table 4. Only tool availability changes within the same Sage-A1.5 model.

Unit: Score

In company tests, BrowseComp, which searches for hard-to-find answers, rose by 1.1 percentage points. The report-generation benchmark DeepResearch-Bench2 rose by 3.8 points. The researchers interpret this as consistent with integrating exploration results into reports.

Table: Evaluation results by verification mode

Conditions: Teacher-model evaluation in technical report Table 5. Output tokens and conversation turns use paired attempts where both modes produced answers.

Units: Score, tokens, turns

Selection criteria

Both modes are off by default. Parallel execution requires registering the spawn_subagent tool and a harness to manage execution. Subagents cannot see the main conversation, so the question, background, and desired output format must all be passed to them.

Verification is enabled per request. Below is part of the request body from the model card example. An empty system message is rejected with HTTP 400.

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Find the population of Shanghai and cross-check it."}
  ],
  "chat_template_kwargs": {"verification": true}
}

Parallelization improved search and report scores in the final model, while verification improved accuracy and reduced generation costs in the teacher model.