Processing 10,000 items is different from 10. Things will fail halfway. Plan for it.
1. Checkpoint every result
Write each result as soon as you have it, and skip items already done:
import json, os
DONE = "results.jsonl"
done_ids = {json.loads(l)["id"] for l in open(DONE)} if os.path.exists(DONE) else set()
todo = [x for x in items if x["id"] not in done_ids]2. Limit concurrency
import asyncio
from openai import AsyncOpenAI
client = AsyncOpenAI(base_url="https://devsaio.lol/v1", api_key=KEY, max_retries=3, timeout=120)
sem = asyncio.Semaphore(8)
lock = asyncio.Lock()
async def run(item):
async with sem:
try:
r = await client.chat.completions.create(model="claude-haiku-5-5", temperature=0,
messages=[{"role": "user", "content": PROMPT.format(**item)}], max_tokens=300)
row = {"id": item["id"], "ok": True, "out": r.choices[0].message.content, "usage": r.usage.model_dump()}
except Exception as e:
row = {"id": item["id"], "ok": False, "error": str(e)}
async with lock:
with open(DONE, "a") as f:
f.write(json.dumps(row) + "\n")
async def main():
await asyncio.gather(*(run(x) for x in todo))
asyncio.run(main())3. Re-run failures
Failed rows are recorded with ok: false. Delete them from the file (or filter them) and run the script again; only failures are retried.
4. Track cost as you go
Sum usage from the results to estimate cost before the job finishes, and give the job's key a monthly limit as a safety net.
5. Test on 20 items first
Run a small sample, check quality and cost per item, then multiply. It's the cheapest way to avoid an expensive surprise.
Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.