4ROUTER / Routing
When a route needs a fallback
Defining alternate paths and the limits of recovery.
Engineering note
Classify the failure first
Authentication errors, invalid inputs, rate limits and temporary unavailability should not share one retry policy. Before falling back, establish whether the failure is recoverable and whether the alternative meets interface, context and task requirements.
Streaming changes the recovery boundary
Changing execution before output begins is relatively straightforward. Once partial output reaches a caller, appending another model’s continuation can break meaning or protocol. Define errors after streaming starts so clients know whether the result is complete.
Keep each attempt visible
Eventual success can still include additional cost and delay. Record selection, attempts, reasons and outcome to explain the recovery path. Bound retry count, overall timeout and budget so an outage does not amplify demand.
Before implementation
- Error classes and recovery conditions
- Post-stream-start handling
- Retry, time and cost boundaries
