Transformers v5.23.0: New model families, unified export

1. Sharing and rerunning exported models now follows a unified workflow

The new export workflow goes beyond creating deployment models to cover saving, sharing, and execution. The resulting ExportedModel object accepts arguments like the original model and returns the same output class. Developers can upload it to the Hub, download it again, and use it in the corresponding runtime.

The original example below shows how to export and share a model for ONNX Runtime after preparing the model and inputs. The final call uses the same form as calling the model before export.

exported = OnnxExporter().export(model, inputs, OnnxConfig(dynamic=True, ort_fusions="cuda"))
exported.push_to_hub("my-org/embeddinggemma-2-onnx")

exported_model = ExportedModel.from_pretrained("my-org/embeddinggemma-2-onnx", device="cuda")
outputs = exported_model(**inputs)  # same output class as model(**inputs)

Exported artifacts can also be stored in a model repository subfolder. export.json records settings, input shapes, package versions, and input and output names. Quantized models with reduced weight precision can be saved and run in the same way, so these models can also be shared.

2. Support expands for multimodal, coding, and judging models

Thinking Machines Lab's Inkling 2 Small takes text and images and outputs text. Its mixture-of-experts architecture selects only some of the model's experts for computation on each input. It has 594B total parameters and 17B active parameters.

Liquid AI's LFM3 is a language model that combines convolution with attention, a method for calculating relationships between input elements. LFM3-VL is a model family that adds image and video processing capabilities.

Dots 3 Note Preview is a mixture-of-experts model that accepts text, image, video, and audio inputs.

JEV, a judging head, was also added. This component scores single-token labels based on the prompt's final token.

IQuest-Coder-V1 is a code model family, while InternLM3 is an instruction-tuned language model. Both can be loaded through auto classes using the existing Llama implementation, reducing the need to learn a separate execution implementation for each model.

3. Execution speed depends on the backend and baseline

During export, multiple operations are fused into a single operation suited to the runtime to reduce overhead. ONNX Runtime applies this to attention and normalization, which adjusts the scale or distribution of values. OpenVINO uses its own CPU kernels, low-level code that directly performs operations, to handle attention and compressed weights. The company reports that the decode stage, which generates successive output, is 2.4~3.9x faster than before.

The performance example uses EmbeddingGemma2 with an input of 8 × 128 tokens. According to the company, ONNX Runtime using the TensorRT execution path on an A100 was 7.6x faster than PyTorch eager execution and 1.4x faster than torch.compile.

With the same model and input on a Xeon Platinum 8570, the company reports that OpenVINO was 4.1x faster than eager execution and 2.2x faster than compiled execution. With int8 weights, the gains were 4.9x and 2.6x, respectively. These results show that expected gains depend on whether a team previously used eager or compiled execution on each device.

4. Adapt return values and saving methods to the new interface

When migrating existing export code, start by checking how results are received. The return value changes to ExportedModel, and the ONNX program uses the onnx_ir.Model format.

The output_path option for specifying a save location has been removed; use save_pretrained to save instead.

Register custom execution backends with register_backend.

IQuest-Coder-V1 Loop checkpoints, model artifacts that store training state and use a recurrent structure, are not supported, and judging heads and pipelines are experimental. The performance figures come from different hardware. The decode improvement range cited earlier was not accompanied by model, input, or hardware details.

5. Integration benefits come with code migration costs

The unified workflow lets other developers receive exported artifacts and run them with the same calling pattern. Existing users, however, must bear the cost of migrating return types and saving code. Teams with multiple deployment targets can assess the benefits of reusing the sharing workflow. Teams with automated exports should first review the interfaces that are changing.

This release expands model choices and helps reuse deployment artifacts, while adoption hinges on runtime-specific benefits and the cost of migrating existing code.