Nemotron-pretraining-Indic-v1: 2.8 trillion-token corpus, open data generation pipeline

1. 22 languages and two script formats

Nemotron-pretraining-Indic-v1 covers the 22 languages listed in India's constitution. A corpus is a collection of training text, and tokens are the units a model uses to process text. All 22 languages have native-script data, and 12 also have romanized data in Latin script, giving 34 language-script combinations.

Counted with the Gemma-4 tokenizer, native-script data totals 2.601 trillion tokens and romanized data 224.6 billion tokens. These figures describe stored data, not the budget for a particular training run. Native-script data is also translated from English. Romanized data only changes the script of some accepted translations, so it does not represent new content.

2. Translate, evaluate, then revise where needed

The starting point is 15 subsets of Nemotron-CC-v2 containing web documents, math, code, and more. NeMo Data Designer connects translation, quality checks, and targeted revisions while preserving source document identifiers for traceability.

The process for 12 languages translates entire documents with Gemma-4-31B-it and evaluates them with the same model. Only scores of 75 or above on a 100-point scale assessing meaning, fluency, terminology, formatting, and other criteria are accepted. This does not mean 75% accuracy; results below the threshold are revised using evaluation feedback and assessed again. Some accepted translations are converted to Latin script and checked separately.

The remaining 10 languages use Bodhan AI's Indic Translate and the same 75-point threshold, but without feedback-based rewriting. Long Santali documents are translated in overlapping segments and joined together. Manipuri is translated into Bengali script, then converted to its native script. This process produces only native-script data.

3. Gains come from a mixed-data training stage

Nemotron 3.5 Super VL continued pretraining on a mix of the corpus's 12-language portion and locally authored text. Of the total 100 billion-token budget, 10%, or 10 billion tokens, went to this mix. The remaining 90% maintained the relative proportions of the existing training data.

Comparing results immediately before and after this stage shows gains of 4.98–9.82 percentage points across five knowledge and reasoning evaluations. For example, Indic MMLU-Pro, evaluated across 12 languages, rose from 51.83 to 56.81. Language coverage varies by evaluation, and the results do not isolate the contribution of synthetic data alone.

Separately, comparing the final model with the previous generation, Nemotron 3 Super, shows improvements on five of six tasks. However, IFEval 1.5, an instruction-following evaluation, fell by 2.04 percentage points. This comparison also reflects subsequent training.

4. Start with a small sample and a fixed budget

Choose the languages and scripts you need, then check the dataset card and license first. The following excerpt from the original example checks the repository and dataset splits. The configuration value denotes Hindi in Devanagari script.

from datasets import get_dataset_split_names, load_dataset

dataset_id = input("Repository ID from the dataset card: ").strip()
config = "hi-Deva-IN"

splits = get_dataset_split_names(dataset_id, config_name=config)
print("Available splits:", splits)

The source recommends streaming to read samples without a full download and pinning the dataset revision after release. For initial experiments, fix the token budget, compare mixing ratios, and record results by language. When splitting training and evaluation data, take care to keep translations and script variants of the same source from appearing on both sides.

5. Automated generation does not remove the duty to validate

The open generation tool, the Nemotron Bharat plugin, lets users apply their own documents and translation guidelines. While this makes it easier to expand technical content across languages, users must define the filtering policy for accepting generated candidates as training data. The source recommends reviewing a small sample with native speakers before scaling up.

Quality assessment is largely automated, and the extent of human sample review varies by language. Errors may remain, and source identifier checks eliminate duplicate processing, not semantically similar documents. Translated data cannot replace locally authored text containing regional knowledge and cultural context.

This release's value lies in providing both data volume and a reusable generation process, while adoption depends on use-specific validation after mixing it with locally authored data.