Mage-VL-Cowork 4B: Screen and terminal integration, staged training recipe

1. One model chooses between screen interaction and code

Mage-VL-Cowork has 4 billion parameters. It uses a mouse and keyboard for tasks requiring visual checks and can use code for structured files such as documents or spreadsheets. The model chooses each tool without a separate orchestrator.

Microsoft released the weights and training code, licensing the model under Apache 2.0. The technical report was published on October 29, 2026.

2. Five stages develop the two abilities separately, then combine them

Training starts from an intermediate model that understands images and text. It proceeds through grounding mid-training, GUI supervised training, coding and terminal supervised training, hybrid supervised training, and online RL.

The design trains screen and terminal skills separately before merging them into one model. After supervised learning from demonstrations, it refines actions using task success as the reward.

The team first refined the teacher model's tools, instructions, and execution environment for generating demonstrations, removing overlapping tools.

The Mage-CUA-Forge data generation system created tasks through reusable workflows. Microsoft reports 293,806 desktop and web tasks and 318,837 verified execution traces based on 6,915 skills.

In Microsoft's comparison using the same generation model, recurring generation costs per task fell from $0.906 to $0.158 versus agent-based task creation. Agents are programs that use individual tools to perform tasks.

3. It scored 66.0% in a 50-step evaluation

Microsoft reports 66.0% on OSWorld-Verified, which evaluates desktop tasks. This rounded mean of 65.33% and 66.75% covers 361 tasks excluding Google Drive, with a 50-step budget and a 1920×1080 screen.

Below are results for models Microsoft listed with a 50-step budget in the same evaluation table.

Table: OSWorld-Verified results with a 50-step budget

Conditions: OSWorld-Verified, reported step budget of 50. Mage-VL-Cowork: 361 tasks excluding Google Drive, 1920×1080 screen, average of 2 runs.

Unit: %

Microsoft reports that ScaleCUA-Qwen3.5-9B led in overall score. Mage-VL-Cowork scored 87.5% on operating system tasks and 62.7% on tasks spanning multiple applications.

4. Calls and output tokens fell after reinforcement learning

Microsoft reports that reinforcement learning improved task scores while reducing average model calls and output tokens versus hybrid supervised training. The table shows performance and generation volume at each training stage.

Table: Evaluation and generation volume by training stage

Conditions: OSWorld-Verified, 361 tasks excluding Google Drive, 50-step budget, models at the end of each training stage

Units: score %, call count, output tokens

Microsoft reports gains of 10.16 percentage points, 13.0% fewer calls, and 25.4% fewer output tokens with RL versus hybrid supervised training. Hybrid supervised training raised scores but used more output tokens. The savings also required final RL.

5. Start local execution in an isolated environment

The Hugging Face model card provides a container command to launch an inference server. Its custom architecture requires allowing remote code, the executable code shipped with the model, when loading it with Transformers.

export MODEL_DIR=/path/to/Mage-VL-Cowork

docker pull xiaoyizhang/mage-vl-cowork:0.1.0

docker run --rm --name mage-vl-cowork \
  --gpus 'device=0' --shm-size=8g \
  -p 127.0.0.1:30001:30001 \
  --mount "type=bind,src=${MODEL_DIR},dst=/models/Mage-VL-Cowork,readonly" \
  xiaoyizhang/mage-vl-cowork:0.1.0

The model can execute arbitrary shell commands. Microsoft recommends disposable sandboxes, such as containers or snapshot-restorable virtual machines, and human approval for hard-to-reverse actions such as external transfers or logins.

Mage-VL-Cowork broadens a small model's uses through a training design that develops screen interaction and code execution separately before combining them, rather than simply adding more tools.