MiniCPM-OCR Jev: Judgment AUROC 93.6, OCR review support with a 2B model

Results

According to OpenBMB's model guide, MiniCPM-OCR Jev achieved an AUROC × 100 of 93.6 on JevOCRBench. AUROC measures how well it separates correct and incorrect answers. The small 2B model follows existing OCR to assist review and candidate reselection. The two tables below split the results from the official PNG into Jev-family and general models.

A separate official SVG labels the Jev family as using a 'Noul' setting, without details. General models use judgment-criteria instructions, so evaluation settings matter when comparing families. The SVG's MiniCPM row reads 93.5·94.5·93.1·91.0, unlike the PNG. The tables below use only PNG figures.

In parameter notation, B means billion, and A denotes the number of active parameters.

Table: Overall and category-level judgment performance on JevOCRBench — Jev family

Conditions: 2,267 JevOCRBench cases, results from OpenBMB's official PNG

Unit: AUROC × 100

Table: Overall and category-level judgment performance on JevOCRBench — general models

Conditions: 2,267 JevOCRBench cases, results from OpenBMB's official PNG

Unit: AUROC × 100

Measurement method

Evaluation used OpenBMB's JevOCRBench dataset. Its 2,267 Chinese and English cases comprise 47.7% text, 24.7% formulas, and 27.6% tables. Each record includes the original region image, candidate text, a rendered candidate image, and a human correctness label. Incorrect candidates were drawn from actual errors by top-tier OCR models and labeled by humans. Scoring uses three fixed inputs: original region image → candidate text → rendered candidate text image. The metric is AUROC, which measures separation of correct and incorrect answers.

OpenBMB's cross-parser evaluation found judgment performance largely stable across parsers, tools that structure OCR results. Four parsers each processed the same 200 randomly selected JevOCRBench samples, and humans labeled correctness.

Table: Judgment performance by parser

Conditions: The same 200 randomly selected JevOCRBench samples, four parsers run independently, OpenBMB's internal evaluation

Unit: AUROC × 100

Results across the four parsers stayed within a 1.5-point range.

Why the results differ

MiniCPM-OCR Jev was trained from Qwen3.5-2B, retaining its vision encoder, which converts images into features, and its language backbone, the core language-processing network. It replaces lm_head with a binary classification head for correct versus incorrect answers.

Scoring has 3 stages: precheck → candidate text rendering → model computation. The precheck validates LaTeX syntax for formula blocks. Candidates are rendered as images for side-by-side comparison with the original region image. OpenBMB's deployment documentation explains that rendering lets the model see candidate text layout.

OpenBMB reports improved parsing quality over each parser alone when using the same document segmentation method and reselecting candidates. MiniCPM-OCR Jev gained 0.18 points for MiniCPM-OCR 1.0, 0.02 for Xiaomi OCR, and 0.25 for PaddleOCR-VL 1.6. These +0.02~+0.25 points are gains in overall document parsing scores, not judgment AUROC. The same judgment model did not yield uniform gains across parsers.

Table: Parsing quality gains from Jev candidate reselection

Conditions: Three parsers share OmniDocBench v1.6's PaddleOCR-VL SDK block segmentation and assembly rules. Each parser alone is the baseline, with low-confidence blocks reselected from the other two models' candidates. Source: OpenBMB's official figure.

Unit: Point improvement

Selection criteria

Set the automatic approval threshold by checking both approval rate and accuracy against human labels on your own data. The official example figure uses 900 human-labeled block images from internal documents. It calculates accuracy at different review-free approval rates. The figure shows 85.0% for legal contracts at a threshold of 0.984, 56.2% for financial statements at 0.995, and 35.1% for mixed documents at 0.994.

The official Docker/SDK local deployment guide requires an NVIDIA GPU with at least 12GB VRAM and driver 535 or later. Disk requirements are about 5~9GB for model computation, about 1.5GB for rendering, and about 2.5GB for weights.

Use the Docker image if you plan to compare absolute scores directly with published figures. OpenBMB's deployment documentation says its rendering matches the cloud API. The locally installed SDK renderer preserves score rankings, but absolute values may differ slightly.

In responses, first check whether verdict, the judgment status, is scored. Treat 0.0 and -1.0 as fixed values returned on scoring failure.

category must match the actual block type.

The following translates the deployment documentation's single scoring request into Python. Use it with the local service running and block.png containing the relevant text region image.

import base64
from pathlib import Path
import requests

response = requests.post("http://127.0.0.1:8030/v1/rm/score", json={
    "input_form": "ternary",
    "items": [{"case_id": "blk-001", "category": "text",
               "image": base64.b64encode(Path("block.png").read_bytes()).decode(),
               "ocr_text": "the OCR text to be scored"}]
})
response.raise_for_status()
result = response.json()["results"][0]
if result["verdict"] == "scored":
    print(result["prob_yes"])
else:
    print("Scoring failed:", result["verdict"])

After deployment, compare scores for the 3 examples with the reference scores and check that differences are within 0.02.

Final candidate reselection quality depends on the range and accuracy of candidates already available.

Weights and code are released under Apache License 2.0, with original and quantized weights available. Deployment supports the cloud API, Docker, and SDK.

A small judgment model can support review and reselection of existing candidates rather than rerunning OCR.