NVIDIA Physical Plausibility Evaluator: Single-video screening, customizable decision criteria

1. What it evaluates

Physical Plausibility Evaluator freezes a roughly 300 million-parameter encoder that extracts video features, training only 2,049 parameters to classify those features as good or bad. Rather than generating videos, it learns human quality labels to evaluate output.

2. Inputs and decision process

Input must be an MP4 video encoded with H.264/AVC. The evaluator uniformly samples 64 frames across the video, including both endpoints. It resizes them to fit 384×384 while preserving the aspect ratio, then pads the remaining space. Videos with fewer than 64 readable frames are rejected; audio and transparency information are not used.

The model uses a badness score to produce a plausibility score in the 0–1 range and a good-or-bad classification. The score is not guaranteed to be a calibrated probability.

The default loads a threshold set to reject 30% of good videos in the validation set; it does not use a score cutoff of 0.3. Users can retrain the classifier or simply recalibrate the threshold to match their criteria.

3. Data and measured results

Classifier training data consists of videos generated with Cosmos 3's image-to-video feature. Quality checks excluded 13 of the original 5,000 videos, leaving 4,987. Human quality ratings on a three-level scale were merged into two categories: good and bad. The training set contains 2,991 videos, totaling about 7.7 hours. Validation and test sets each contain 998 videos, with no guarantee that the same scene does not appear in different splits. The data itself and detailed generation settings are not disclosed.

The pretrained classifier was evaluated on 998 test videos using an NVIDIA Hopper H100. Its ROC AUC, measuring its ability to rank bad and good videos by score, was 0.8258. No comparisons with other models were provided.

A raw classifier output threshold of 0.5438173 was set on validation data for a 30% cap on good-video rejection. Applying it filtered out 562 of 698 bad videos, yielding 80.52% recall. Of 300 good videos, 94 were rejected, a rejection rate of 31.33%. Test labels were not used to select the threshold.

The benefit of filtering out bad videos comes at the cost of discarding good output too. Since the validation rejection cap differs from the actual test rejection rate, decision criteria must be checked again on deployment data.

4. Deployment considerations

The model card describes command-line and REST API integration. However, this model is absent from the check list in the provided Cosmos Evaluator repository documentation. Below is the repository cloning example from the source; no command for running this evaluator was provided.

git clone https://github.com/NVIDIA/cosmos-evaluator.git
cd cosmos-evaluator

Video feature extractor weights must be obtained separately and used under their original distribution terms.

The model card does not certify GPU architectures beyond the validated hardware. The encoder and classifier run on the GPU, while video decoding and resizing run on the CPU.

There are no plans for ongoing retraining or maintenance. Deployment planning must also consider how to operate the model after in-house adjustments.

This model does not identify where or when specific defects occur. A passing result certifies neither physical accuracy nor safety, so it must not be the sole basis for safety-critical decisions.

This model helps screen synthetic videos, but it does not take over responsibility for verifying decision criteria and suitability for use.