FLUX 3 Image Klein Base: 3.46x generation, eight-GPU scaling bottlenecks

Measurement method

On November 6, 2026, NVIDIA released a tutorial for VisualGen, TensorRT LLM's dedicated diffusion model runtime. It covers Base and a model trained to use fewer generation steps. This 16-step model achieved up to 2.34x end-to-end speedup.

End-to-end latency is generation pipeline time excluding prompt expansion and upsampling. Lower LPIPS means closer to the baseline output. First, we examined time spent denoising, which progressively removes noise. T2I means text-to-image generation, and I2I means image editing.

Table: FLUX 3 Image Klein pipeline latency by stage

Conditions: One B200, 2048×2048, NVFP4, VANILLA attention, and torch.compile. Attention selects and references information across parts of an image. Base: 50 steps, CFG 3.5; step-reduced model: 16 steps, no CFG. CFG adjusts conditioning strength using positive and negative prompts. Average of 10 requests after 2 warmups; NVIDIA's own measurements.

Units: ms, s

Results

Table: Single-GPU end-to-end latency before and after optimization

Conditions: One B200, 2048×2048, Base. Uncompiled BF16 VANILLA versus compilation+NVFP4 PrimsTS+Sol-Attention+Cache-DiT. LPIPS values compare example images with baseline outputs; NVIDIA's own measurements.

Units: s, x

Table: End-to-end latency and denoising gains with multi-GPU scaling

Conditions: DGX B200 scaling from 1 to 8 GPUs, 2048×2048, NVFP4 T2I. Base uses 2 CFG groups and Ulysses 4; the step-reduced model uses Ulysses 8. NVIDIA's own measurements.

Units: s, x

Why the results differ

The individual techniques below use NVIDIA measurements on one B200 at 2048×2048 with Base NVFP4 T2I. BF16 attention in the default VANILLA backend took about 60% of GPU kernel time in the diffusion transformer (DiT). The DiT progressively refines images.

Quantization uses NVFP4, Blackwell's low-precision format. On one Blackwell GPU, generation was up to 1.62x faster for T2I and 1.58x for I2I than BF16. It loads BFL's prequantized DiT checkpoint, so no separate quantization is needed. FlashInfer's PrimsTS kernels handle matrix multiplication. NVIDIA measured T2I speedups of 1.02~1.10x over an autotuned TensorRT LLM NVFP4 GEMM implementation on B200.

Against VANILLA BF16 attention, cuDNN FP8 delivered 1.08x generation speedup. VC-Attention FP8 (ρ=0.005), which corrects only tokens with large errors, delivered 1.18x. The latter's example LPIPS with CUDA Graphs enabled was 0.086.

Sol-Attention (τ=1.0) skips unnecessary computation. Dense computation for the first 16 steps delivered a quality-first 1.24x speedup; 12 steps delivered a speed-first 1.30x. These are denoising plus autoencoder decoding speedups over compiled dense VANILLA. With 12 steps and CUDA Graphs enabled, example LPIPS was 0.151 for a car scene and 0.140 for a cat scene, from different images. Raising τ further reduced quality without much additional speed gain.

Cache-DiT reuses intermediate transformer outputs across steps without modifying weights. At a residual difference threshold of 0.24, which sets the tolerance for output changes, generation was 1.94x faster than without caching, with example LPIPS of 0.033.

With multiple GPUs, Base combines CFG parallelism, which splits positive and negative prompts across GPU groups, with Ulysses parallelism, which splits the sequence. The step-reduced model uses only Ulysses.

In the 2048×2048 scaling table above, end-to-end gains are 14.90/3.00≈4.97x for Base and 2.73/1.08≈2.53x for the step-reduced model, below the denoising gains of 6.01x and 4.44x. With eight GPUs, the step-reduced model still spends about half its time outside denoising, so overall response speed does not scale with GPU count.

A separate 1024×1024 NVFP4 VANILLA experiment added CUDA Graphs to compilation to record and reuse kernel execution. This reduced launch overhead as work per GPU decreased. NVIDIA measured Base at 1.823→0.809s and the step-reduced model at 0.791→0.366s on eight B200 GPUs.

Selection criteria

To reduce response time on one GPU, start with an NVFP4 checkpoint, torch.compile, and Cache-DiT, then add Sol-Attention after checking quality. If almost no quality loss is acceptable, omit Sol-Attention. Choose the Cache-DiT and NVFP4 combination by comparing real workload images with baseline outputs.

In a separate experiment on different hardware, NVIDIA reports 1.73~3.25x speedups on one RTX PRO 6000 Blackwell Server Edition using compilation, CuTe DSL NVFP4, and Cache-DiT.

To select NVFP4, add the following to YAML. Follow the deployment guide below for installation, execution, and other settings.

pipeline_config:
  variant: nvfp4

The comparison uses NVIDIA's own measured server environment; the same ratios are not guaranteed on other GPU configurations.

As denoising bottlenecks shrink, time outside denoising constrains scaling; evaluate end-to-end latency alongside LPIPS rather than GPU count alone, while remembering that VisualGen is still in beta.