LocalAI 4.11.0, released October 2, adds failover between models. A single model name can point to an ordered list of local and remote targets.
Transport errors, server failures, out-of-memory conditions and rate limits can move a request to the next target before a response is committed. Invalid input, ordinary client errors and cancellation do not.
GET /api/failover exposes the chain and target health. Administrators can pin or unpin targets through the API, MCP tools or React interface, and X-LocalAI-Served-Model identifies the model that answered.
Recovery probes and a minimum fallback residence period reduce repeated switching between targets.
Failures after streaming starts and other additions
Once a response stream has started, failover cannot recover a later error. The calling application still needs to decide how to handle that case.
The release also adds OCI model galleries with Sigstore verification, a single-host operations view, and transcription, diarization and sound detection. Speaker names live only in the running process and disappear on restart; speaker profiles are treated as sensitive biometric data.
Two boundaries to inspect in operations
Remote fallback can improve availability while allowing request data to leave the local environment. Trust boundaries for fallback targets are part of the configuration decision.
Checking a failure before the response and another during streaming can expose the boundary between LocalAI failover and application recovery. These are checks suggested from release documentation, not tests performed for this article.
LocalAI 4.11.0 handles selected failures before output begins; later recovery and the data path remain separate design decisions.