NVIDIA released a method that automatically scores model answers against experimental records and uses those scores as rewards for post-training on scientific tasks. Researchers can connect their experimental data to verification tools to build domain-specific models. The drug metabolism prediction example applies supervised fine-tuning followed by reinforcement learning.
1. Turning experimental results into automatic scoring criteria
The recipe takes experimental data with recorded measurements as input. The example stores molecular structures, reaction conditions, and experimental results in a CSV, turning each row into a question and reference answer for the model. Reference answers come from experimental records rather than human preference labels and serve as reward signals.
The example task predicts the metabolites produced from a parent molecule and a metabolic enzyme. It uses MetxBioDB, which contains experimentally confirmed biotransformations, or chemical reactions within organisms. The scorer compares molecular fingerprints, numerical representations of structural features. It measures agreement between predictions and reference answers using Tanimoto similarity, a measure of structural overlap. Scores range from 0 to 1, with higher values indicating more similar fingerprints.
Verifiers can be tailored to the task, from simple similarity calculations to cheminformatics tools, molecular simulators, and autonomous laboratories. Cheminformatics tools compute and analyze chemical information. A key design choice is the reward function: the rule that defines success and turns results into scores. Rewards must accurately reflect scientific outcomes for the model to learn in the intended direction.
2. Reinforcement learning improves answers through automatic scoring
After setting up the environment, NVIDIA describes how to tune Nemotron 3.5 Super VL with reinforcement learning. The model reasons over the input and calls tools, while the verifier evaluates submissions and returns rewards. This example uses Morgan fingerprints with a radius of 2 and a length of 2048 bits, scoring only the first submission.
According to the company, supervised fine-tuning (SFT) on reference examples improved accuracy by 12% in the drug metabolism prediction environment. This used an output budget, the token limit for model responses, of 16,384 tokens. Reinforcement learning (RL), which adjusts responses using scoring rewards, added another 17%. A figure in the blog also shows average Tanimoto similarity at each post-training stage.
Table: Average molecular similarity in drug metabolism prediction
Conditions: Company-published figure, drug metabolism prediction environment, Morgan fingerprint Tanimoto similarity
Unit: Tanimoto similarity
| Model and training stage | Average similarity |
|---|---|
| Nemotron 3.0 Super | 0.33 |
| Nemotron 3.5 Super VL | 0.69 |
| Nemotron 3.5 Super VL, after SFT | 0.78 |
| Nemotron 3.5 Super VL, RL post-training from the SFT checkpoint | 0.90 |
The figure labels the change from 0.69 to 0.90 as a 31% improvement. This is a company-reported result for a specific drug metabolism task. Its metric differs from the accuracy gains reported under the 16,384-token condition.
3. Experimental data and infrastructure underpin performance gains
The published example runs multi-node GRPO on 4 nodes with 4 GPUs per node. GRPO learns from reward differences within groups of responses. It uses NeMo Gym for the environment and NeMo RL for post-training, with steps from data preparation to connecting the reward server and training tools. The same structure can support other tasks that return numerical rewards, but each domain needs task-specific experimental data and scoring tools.
# Multi-node GRPO run (16 GPUs)
python examples/nemo_gym/run_grpo_nemo_gym.py \
--config configs/grpo_metxbiodb_super_vl.yaml \
cluster.num_nodes=4 cluster.gpus_per_node=4 \
grpo.max_num_steps=100These gains require researchers to organize experimental data and design verifiers, and the example configuration uses multi-node resources totaling 16 GPUs. The optional teacher-model path also requires model inference calls. Adoption therefore starts with checking whether the task has clear reward criteria that data can verify. Comparing against the starting model on data not used for training can reveal whether post-training improved performance.
4. Teacher distillation is an optional preparation step for RL
If the base model lacks capability or RL converges slowly, distillation can be added before reinforcement learning. This method uses teacher answers as training data. A stronger teacher model generates reasoning examples in the same environment, and the student model first fine-tunes on high-scoring examples. The teacher supplies answers and training data, while the student learns from those examples. The company says this warm start can speed convergence, when training results stabilize, and make subsequent RL more efficient.
In the drug metabolism example, Anthropic's Claude Sonnet 5.5 serves as the teacher model. From 1,934 training prompts, 1,123 reasoning examples with Tanimoto scores of at least 0.7 were selected for SFT. Raising the threshold, the score that determines inclusion, retains only more closely matching answers. Lowering it can admit variants that differ from the reference answer.
The core of this recipe is not the model itself, but verifier design that turns experimental results into usable training rewards.
