Apple researchers released two models that rapidly extend videos in response to user input, along with code and intermediate training checkpoints. Developers can download them to generate videos from text or try demos where scenes change as the camera moves.
1. Replacing two costly operations with lightweight components
FastFrames is an autoregressive video generation model that creates the next scene from earlier scenes and new input. Rather than making users wait for a finished video, it extends scenes in response to input. Proposed uses include reflecting camera motion and physical interactions in video.
The researchers found that decoding compressed internal representations into images and updating cached frame information can account for more than half of existing models' total latency. They replaced these operations with a lightweight pixel decoder and a cache refinement network, a neural network that refines prior-frame information. System optimizations also accelerated diffusion, the process of generating video by gradually reducing noise.
2. Shorter waits than existing models on the same single GPU
The company reports the following latency per generation unit for 480p video on one NVIDIA B200 GPU, in steady state after startup. Each model size has a comparison model, showing the latency reduction at a similar scale.
Table: Latency per generation unit on one B200
Conditions: 480p, 1 NVIDIA B200 GPU, steady state, per 1 generation unit
Unit: ms
| Model | Latency |
|---|---|
| CF++ | 272 |
| FastFrames-S | 36 |
| Krea-Realtime | 1,675 |
| FastFrames-L | 158 |
The company reports that FastFrames-S is nearly 8x faster while maintaining the generation quality of Causal Forcing++.
The company says the larger FastFrames-L is over 10x faster while offering realism, instruction following, composition, and text rendering comparable to Krea-Realtime.
For 480p video, the paper's abstract reports 36 milliseconds per frame and 28 responses per second for the small model on one B200. The large model achieves 45 milliseconds per frame and 22 responses per second on an 8-B200 node. These response rates are representative results targeting user-controlled video experiences.
3. Both models released alongside training checkpoints
FastFrames-S has 1.3 billion parameters, the adjustable values learned during training, while FastFrames-L has 14 billion. Both generate 480p video in two diffusion steps. Offering two model sizes at the same resolution lets developers test responsiveness and video quality.
The release goes beyond the final generators, decoders, and cache refinement networks. The large model's repository also includes teacher models, whose outputs serve as examples, and intermediate checkpoints from distillation, which learns from those outputs. Researchers can inspect models at different training stages, not just the final results.
An optimized version of LingBot 1.3B was also released. The company reports that it is 16x faster than the official model. The project page shows side-by-side responses to keyboard camera movement with the same inputs supplied simultaneously.
4. Install locally to generate video or run interactive demos
The repository assumes Linux x86-64, an NVIDIA GPU, and a CUDA 12.8-compatible driver. Set up the environment at the repository root and follow the checkpoint instructions to download the weights, the model's learned internal values. Then generate a video with this command.
uv run python scripts/generate.py --model ff-s --checkpoint-dir checkpoints \
--prompt "A corgi surfing a wave at sunset" --seed 0 --output outputs/corgi.mp4The default example outputs an 832×480 video with 81 frames at 16 frames per second. Here, 16 frames per second is the playback speed of the saved video.
The released demos show keyboard camera controls and physics-based dragging, which moves objects according to physical laws. Developers can first check the output with the video generation example, then follow separate setup instructions to run the interactive demos.
5. Fast responses from the large model require more GPU resources
The speed improvements reduce the wait for the next scene after user input. The company reports that the large model drops from 158 milliseconds on one B200 to 45 milliseconds on eight. Shorter waits therefore come with the resource cost of using multiple GPUs.
The small model's single-GPU result and the large model's eight-GPU result use different hardware configurations, so they cannot directly rank model speeds. Developers must choose a configuration based on both their target video quality and the number of GPUs available.
FastFrames' value lies in pairing a design that reduces video-generation delays with released training artifacts, letting developers test the trade-off between quality and compute resources.
