Vision-capable models can read invoices and receipts directly from images, including messy phone photos.
1. Send the image
import base64, json
def extract(path):
b64 = base64.b64encode(open(path, "rb").read()).decode()
mime = "image/png" if path.endswith(".png") else "image/jpeg"
r = client.chat.completions.create(model="gpt-5.6-luna", temperature=0, messages=[{"role": "user", "content": [
{"type": "text", "text": """Extract this invoice as JSON:
{"vendor": str, "invoice_number": str|null, "date": "YYYY-MM-DD"|null, "currency": str,
"items": [{"description": str, "qty": number, "unit_price": number, "amount": number}],
"subtotal": number|null, "tax": number|null, "total": number}
Use null when a field isn't visible. Reply with JSON only."""},
{"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}},
]}])
text = r.choices[0].message.content.strip().removeprefix("```json").removesuffix("```")
return json.loads(text)For PDFs, convert pages to images first (for example with pdf2image), or extract the text layer if it exists and send text instead, which is cheaper.
2. Validate the maths
Models occasionally misread a digit. Check the numbers add up:
def check(inv):
items = sum(i["amount"] for i in inv["items"])
expected = (inv.get("subtotal") or items) + (inv.get("tax") or 0)
return abs(expected - inv["total"]) < 0.02Send anything that fails the check to a human review queue.
3. Tips
- Ask for
nullinstead of guesses; invented values are worse than missing ones. - Keep
temperatureat 0. - Resize huge photos; very large images cost more without improving accuracy.
Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.