DEVSAIO

Dealing with 429 Too Many Requests

Why rate limits happen, how to stay under them with queues and concurrency limits, and how to retry correctly.

DEVSAIO team · 2 min read
Share on X

A 429 means “too many requests right now”. It's temporary. The fix is to slow down, not to give up.

Why you might see it

  • You're sending many requests in parallel (batch jobs, load tests).
  • An upstream provider is under heavy load and limiting traffic.
  • A single user of your app is spamming a feature.

Retry with backoff

Wait and try again with increasing delays (1s, 2s, 4s, 8s) plus a little randomness. If the response includes a Retry-After header, respect it. See errors and retries.

Limit your own concurrency

For batch jobs, process items with a fixed number of workers instead of firing everything at once:

python
import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="https://devsaio.lol/v1", api_key=KEY)
sem = asyncio.Semaphore(5)  # at most 5 requests in flight

async def process(item):
    async with sem:
        r = await client.chat.completions.create(model="claude-haiku-5-5", messages=[{"role": "user", "content": item}])
        return r.choices[0].message.content

results = asyncio.run(asyncio.gather(*(process(x) for x in items)))

Protect your app from abusive users

If your app is public, limit requests per user (for example 20 per minute) on your server. That protects both your balance and the shared rate limits.

Queue long-running work

For anything non-interactive (reports, bulk summaries), put jobs in a queue and process them in the background. Users get a notification when it's done instead of waiting on a page that might time out.

Try it with one API key

Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.

Keep reading