NVIDIA FOX: Factory response coordination, human review boundaries

1. Adding a manager agent to existing factory systems

FOX is a reference architecture showing how a factory manager agent delegates tasks to specialist agents and gathers results. Like a supervisor assigning inspection and analysis tasks, it monitors operations, investigates defect causes, and coordinates robot tasks.

A Siemens report estimates that unplanned downtime costs an average large factory in the surveyed industries $253 million annually. FOX targets not a lack of data but the time and effort needed to gather evidence across systems and turn it into the next response.

Siemens' electronics factory in Erlangen produces more than 1,000 product types annually and frequently switches production for new products or orders. Production planning for rush orders and inspection preparation for new products were therefore presented as agent use cases.

2. Supporting production planning and new product preparation

In the rush-order case, agents built with the FOX blueprint and NVIDIA Nemotron run simulations in Siemens Intelligence Center X. They adjust plans based on costs, existing orders, supply levels, and production line settings. This reduces the need for planners to query each system separately. According to the company, the full agent system can cut costs by up to 20%.

The new product case combines Siemens' product lifecycle management software and Process Preparation X with NVIDIA Cosmos and Omniverse. Product lifecycle management handles information from design to disposal. Virtual prototypes and synthetic training data help prepare and optimize automated optical inspection before physical production. This inspection uses cameras to find visual defects. According to the company, virtual production preparation can reduce time to market by up to 40%. The rush-order case targets cost adjustments, while the new product case targets shorter inspection preparation times.

3. Roles of the manager agent and specialist agents

The manager agent includes a language model that interprets requests and plans tasks. Language models understand and generate text. A runtime framework connects tools, memory, and specialist agents and manages their execution. An isolation layer controls the execution environment and access policies, separating execution to enforce access controls. Developers can choose commercial or open models to suit deployment needs. NemoClaw is the default runtime framework, but alternatives are supported.

Developers connect databases, production systems, and resource management systems. The manager agent uses this data, manuals, and work procedures to interpret factory conditions.

Specialist agents handle defined tasks such as visual inspection or detecting anomalies in measurements over time, identifying values outside normal ranges. They return results, which the manager agent uses to decide the next response.

4. When board defect investigations pass to humans

Separate from Siemens' production planning case, FOX presents a defect investigation on an assembly line that mounts electronic components on circuit boards. These boards hold and connect components. An optical inspection alert starts the investigation. The manager agent retrieves the board identifier and inspection images and requests a second assessment. The visual inspection agent calls NVIDIA Cosmos to compare images with factory manuals, using examples and instructions in its reasoning.

An assessment of acceptable variation includes evidence supporting continued production. Confirmed defects trigger rework and root cause investigation, while uncertain assessments go to human review.

Transporting defective boards for rework is demonstrated with NVIDIA Isaac Sim and ROS 2 in a factory digital twin. This is a virtual model of actual processes and equipment. Operators repair the boards, reinspect them, and then package them.

The root cause analysis agent gathers inspection results, component placement records, electrical test results, and equipment measurements from the board's process history. This history records each manufacturing stage the product passed through. For suspected soldering defects, it retrieves records based on when the board entered and exited the reflow oven, which heats solder to melt it and then cools it. It compares these with other production records and manuals to investigate the cause. A PDF report compiles supporting records, suspect measurements, probable causes, and recommended actions for the line manager to review.

5. Automation benefits and factory responsibility

Agents can reduce repetitive investigative work by collecting evidence, conducting initial analysis, and coordinating responses. Developers and operators instead focus on linking identifiers, times, and units across processes, setting assessment criteria, validating outputs, and designing access policies.

Connecting physical equipment and robots requires factory-specific integration, and operator approval before board handling is recommended in initial deployments. Fully on-premises deployments also require models and services to run internally. When using external models, teams must review the data being transmitted.

FOX's core is designing what agents execute and when humans must decide, rather than simply handing factory tasks to agents.