Image and video generation tool Diffusers has added five models covering image editing, speech-driven human motion, and scene simulation. Developers can assemble reusable components, while existing users gain inference and saving fixes.
1. Assemble image generation and editing from components
This version's modular pipelines load generation components separately. MiniMax-H3-Image generates and edits images from text and one or more reference images in a fixed 6 steps. A separate flow expands short requests and suggests aspect ratios.
The release notes give the following loading sequence for this model.
ModularPipeline.from_pretrained(...)
load_components(...)Pruna AI's P-H3-Image supports text-based generation and instruction-based editing with up to 8 input images. It includes a component that expands requests into detailed prompts. The released model uses 8 steps. Both models offer request expansion.
2. Generate sound alongside video or animate people to recordings
Based on MiniMax-H3, Pika H3 Flex generates video and synchronized stereo sound, with separate left and right channels, in 6 steps. Separate flows handle text, first or last frames, and various references. Two pipelines can reuse loaded components.
Kandinsky S2V 1.0 creates speech-aligned full-body motion videos from a reference image and voice recording. It generates 5-second segments, passing context forward. The recording supplies audio. The distilled variant, trained to mimic a larger model's output, defaults to 16 steps.
3. Expand scene simulation and device-specific inference options
PAN 2 simulates scenes from prompts describing actions and camera movements over time. It has 29 billion parameters, internal values adjusted during training. It generates video from text or images, with context parallelism across devices and caching to reuse results.
AMD's RDNA4 gfx1201 GPU gains aiter_sage_hub as an inference computation option. It downloads specialized code, or kernels, from the Hub and falls back to default operations for unsupported input shapes. A fix prevents one model's choice from affecting others.
MiniMax-H3 gains tensor parallelism, splitting internal model computations across devices. This gives developers another configuration option for distributed execution of large models.
4. Fix saved files and state management across generations
When multiple LoRA adapters are attached to tune a model, component-specific weight settings are now passed correctly to each adapter.
Some image-generation adapter training examples now correctly apply gradient clipping, which limits excessive updates during training to prevent abrupt changes in learned values.
Saving checkpoints containing trained weights no longer deletes shard files for other variants. Converting shards to a single file removes stale indexes. These fixes affect users managing saved copies of multiple variants.
Modular pipelines reset their computation reuse state on each call. They also clean up after calls interrupted by errors, so the next generation starts with fresh state.
5. More options require more environment preparation
New models and distributed computation require preparing extra dependencies: external software needed to download and run selected kernels. Alongside easier assembly of generation flows, this update calls for checking the execution environment.
Dependency checks now enforce the existing minimum versions, torch 2.6 and transformers 4.41.2, so environments below these versions receive upgrade guidance.
PAN 2's prompt and video safety checks are enabled by default and require the separate pan2-guardrail package.
0.43.0 expands modular support with five models while reducing practical errors in inference, saving, and training.