How does an LLM API fail over when the primary model is unavailable?
Learn the difference between fixed routing and failover, what happens when a primary model is unavailable, and how to prepare controlled candidate resources for model calls in Lumoswitch.
Maintained by the 路墨 Lumoswitch product team
Failover requires configured available candidates and an order in advance; fixed routing does not switch automatically. It reduces the impact of one upstream failure but cannot guarantee permanent model or upstream availability.
What is the difference between fixed routing and failover?
Fixed routing sends a call to a specified resource and does not automatically try another candidate after an error. Failover requires an administrator to configure candidate resources and their order in advance, then continues within that candidate scope when an error meets the runtime policy. Fixed routing emphasizes determinism; failover emphasizes availability preparation.
Do all model failures switch automatically?
No. Switching depends on whether the public model has a failover policy, whether candidate resources are available, and whether the actual error meets runtime handling conditions. Failover cannot repair a model's output-quality problem or guarantee that any upstream service is always available.
How should candidate resources be configured?
First define a stable public model ID for clients, then choose the candidate resources and order allowed to handle that request. Candidates should fit the same use case in model capability, interface compatibility, price, and compliance boundaries. Do not indiscriminately mix models with materially different semantics or capabilities into one failover chain.
How can a problem be located after an error?
Use call logs to confirm request status, duration, actual routing result, and candidate-resource status, then inspect the model scope and rate limits on the model configuration attached to the access key. If the problem is upstream credentials, model health, or candidate configuration, fix it in the console and validate again; do not rely only on client retries to hide a persistent failure.
Can failover call an unauthorized model?
Failover runs only within the candidate resources and routing rules saved by an administrator for that model configuration; it does not cross configured model boundaries. If candidates are not correctly published, unavailable, or unsuitable for the interface, the request can still fail rather than selecting an arbitrary model.