DEVSAIO

Handling AI API errors and retries the right way

Which errors are safe to retry, how to back off, and how to make AI features degrade gracefully instead of breaking.

DEVSAIO team · 2 min read
Share on X

AI APIs fail more often than typical APIs: providers get overloaded, requests time out, and long generations are fragile. Good error handling is what separates a demo from a product.

Retry or not?

ErrorRetry?Why
Network error / timeoutYesUsually transient
429 Too Many RequestsYes, after a pauseRate limit
500, 502, 503, 504YesTemporary provider issue
400 Bad RequestNoYour request is invalid; fix it
401 UnauthorizedNoBad or revoked key
402 / 403NoBalance, key limit or permission problem
404NoWrong path or model id

Exponential backoff with jitter

python
import random, time
import openai

def with_retries(fn, attempts=4):
    for i in range(attempts):
        try:
            return fn()
        except (openai.APIConnectionError, openai.RateLimitError, openai.InternalServerError) as e:
            if i == attempts - 1:
                raise
            time.sleep(min(30, 2 ** i) + random.random())

answer = with_retries(lambda: client.chat.completions.create(model="claude-sonnet-4-6", messages=msgs))

The SDKs also have a built-in max_retries option for the basic cases.

Fall back to another model

With one gateway, a fallback is just a different model name:

python
for model in ["claude-sonnet-4-6", "gpt-5.6-luna", "gemini-3.5-flash"]:
    try:
        return client.chat.completions.create(model=model, messages=msgs)
    except openai.APIStatusError as e:
        if e.status_code < 500 and e.status_code != 429:
            raise

See model routing and fallbacks for a fuller pattern.

Degrade gracefully

  • Show a friendly message and a Retry button, not a stack trace.
  • Save the user's input so they don't have to retype it.
  • Log the status code and error message (never the API key) so you can debug later.
Try it with one API key

Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.

Keep reading