Every LLM provider has occasional outages, and an app that depends entirely on one provider with no fallback plan goes fully dark the moment that provider does — often at the worst possible time.
Levels of graceful degradation
- Best: automatic fallback to a second provider or a smaller local model for critical functionality, so users barely notice.
- Good: a clear, honest error message and a queued retry, so the request isn't silently lost even if it can't complete immediately.
- Minimum: the rest of the app (anything not depending on the LLM) keeps working normally — an AI feature failing shouldn't take down unrelated functionality.
Building toward this
Isolate LLM calls behind a clear interface in your codebase so a fallback provider or a cached response can be swapped in during an outage without touching unrelated code.
See multi-provider LLM fallback.
— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.