AI chat APIs don't remember anything between requests. The “memory” in a chatbot is just your code sending the previous messages again each time.
messages = [{"role": "system", "content": "You are a helpful tutor."}]
def ask(text):
messages.append({"role": "user", "content": text})
r = client.chat.completions.create(model="claude-sonnet-4-6", messages=messages)
answer = r.choices[0].message.content
messages.append({"role": "assistant", "content": answer})
return answerThe cost problem
Turn 1 sends 2 messages. Turn 20 sends 40. Because input is billed every time, the cost of each new message grows as the conversation gets longer.
Strategies
1. Keep a window
Send the system prompt plus only the last N messages:
window = [messages[0]] + messages[-12:]Simple and effective for casual chat.
2. Summarise older turns
When the history gets long, ask a cheap model to summarise the older part, then keep the summary plus recent messages:
summary = client.chat.completions.create(model="claude-haiku-5-5", messages=[
{"role": "user", "content": "Summarise this conversation in 5 bullet points, keeping names, numbers and decisions:\n" + old_text}
]).choices[0].message.content
messages = [messages[0], {"role": "system", "content": "Earlier conversation summary:\n" + summary}] + messages[-8:]3. Store facts separately
For assistants that need long-term memory (user preferences, account details), store facts in your database and add only the relevant ones to the system prompt.
4. Start fresh for new topics
In UIs, encourage a “New chat” for unrelated questions.
Don't forget the assistant's own messages
Always append the model's answers to history. Without them, the model loses track of what it already said.
Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.