RAG (retrieval-augmented generation) lets a model answer questions from your documents without retraining anything: find the relevant passages, then give them to the model with the question.
1. Chunk your documents
from pathlib import Path
def chunks(text, size=800, overlap=150):
words = text.split()
step = size - overlap
return [" ".join(words[i:i + size]) for i in range(0, max(len(words) - overlap, 1), step)]
docs = []
for p in Path("docs").glob("*.md"):
for i, c in enumerate(chunks(p.read_text())):
docs.append({"source": f"{p.name}#{i}", "text": c})2. Embed every chunk (once)
import os, json
from openai import OpenAI
client = OpenAI(base_url="https://devsaio.lol/v1", api_key=os.environ["DEVSAIO_API_KEY"])
def embed(texts):
r = client.embeddings.create(model="bge-m3", input=texts)
return [d.embedding for d in r.data]
for i in range(0, len(docs), 64):
batch = docs[i:i + 64]
for d, v in zip(batch, embed([d["text"] for d in batch])):
d["vec"] = v
json.dump(docs, open("index.json", "w"))3. Retrieve the best chunks
import math
def cos(a, b):
return sum(x * y for x, y in zip(a, b)) / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
def search(q, k=4):
qv = embed([q])[0]
return sorted(docs, key=lambda d: cos(qv, d["vec"]), reverse=True)[:k]4. Answer with the context
def answer(q):
ctx = "\n\n".join(f"[{d['source']}]\n{d['text']}" for d in search(q))
r = client.chat.completions.create(model="claude-sonnet-4-6", temperature=0.2, messages=[
{"role": "system", "content": "Answer only from the provided context. Cite sources in [brackets]. If the answer isn't in the context, say you don't know."},
{"role": "user", "content": f"Context:\n{ctx}\n\nQuestion: {q}"},
])
return r.choices[0].message.content
print(answer("What is the refund policy?"))Making it production-ready
- Store vectors in a real vector store (pgvector, Qdrant, Chroma) once you have more than a few thousand chunks.
- Re-embed only documents that changed.
- The “say you don't know” instruction plus citations is your best defence against made-up answers.
Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.