AI APIs fail more often than typical APIs: providers get overloaded, requests time out, and long generations are fragile. Good error handling is what separates a demo from a product.
Retry or not?
| Error | Retry? | Why |
|---|---|---|
| Network error / timeout | Yes | Usually transient |
| 429 Too Many Requests | Yes, after a pause | Rate limit |
| 500, 502, 503, 504 | Yes | Temporary provider issue |
| 400 Bad Request | No | Your request is invalid; fix it |
| 401 Unauthorized | No | Bad or revoked key |
| 402 / 403 | No | Balance, key limit or permission problem |
| 404 | No | Wrong path or model id |
Exponential backoff with jitter
import random, time
import openai
def with_retries(fn, attempts=4):
for i in range(attempts):
try:
return fn()
except (openai.APIConnectionError, openai.RateLimitError, openai.InternalServerError) as e:
if i == attempts - 1:
raise
time.sleep(min(30, 2 ** i) + random.random())
answer = with_retries(lambda: client.chat.completions.create(model="claude-sonnet-4-6", messages=msgs))The SDKs also have a built-in max_retries option for the basic cases.
Fall back to another model
With one gateway, a fallback is just a different model name:
for model in ["claude-sonnet-4-6", "gpt-5.6-luna", "gemini-3.5-flash"]:
try:
return client.chat.completions.create(model=model, messages=msgs)
except openai.APIStatusError as e:
if e.status_code < 500 and e.status_code != 429:
raiseSee model routing and fallbacks for a fuller pattern.
Degrade gracefully
- Show a friendly message and a Retry button, not a stack trace.
- Save the user's input so they don't have to retype it.
- Log the status code and error message (never the API key) so you can debug later.
Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.