DEVSAIO

Streaming AI responses with Server-Sent Events (SSE), explained

Why streaming makes AI apps feel fast, what the SSE stream looks like, and how to parse it correctly in Python, Node and the browser.

DEVSAIO team · 2 min read
Share on X

Without streaming, your user stares at a spinner until the whole answer is ready, which can be 20 seconds or more for a long reply. With streaming, the first words appear in under a second. The total time is the same, but the app feels far faster.

What the stream looks like

Add "stream": true to a chat request. Instead of one JSON response you get a series of Server-Sent Events:

text
data: {"choices":[{"delta":{"content":"Hel"}}]}

data: {"choices":[{"delta":{"content":"lo!"}}]}

data: {"choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Each data: line is a JSON chunk with a small piece of text in choices[0].delta.content. [DONE] marks the end.

Parsing it correctly

The most common bug: assuming each network chunk contains whole lines. It doesn't. A JSON line can be split across two reads. Always buffer:

js
const res = await fetch("/api/chat", { method: "POST", body: JSON.stringify({ messages }) });
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop(); // keep the incomplete last line for the next read
  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6).trim();
    if (data === "[DONE]") return;
    const json = JSON.parse(data);
    output.textContent += json.choices?.[0]?.delta?.content ?? "";
  }
}

With the SDKs it's simpler

Python:

python
for chunk in client.chat.completions.create(model="claude-sonnet-4-6", messages=msgs, stream=True):
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

The SDKs handle buffering for you.

Proxying streams through your server

Your API key must stay on the server, so the browser usually streams from your endpoint, which streams from the AI API. Make sure nothing buffers in between:

  • Send Content-Type: text/event-stream (or text/plain) and flush after each write.
  • With nginx, add the header X-Accel-Buffering: no.
  • Disable compression on the streaming route; gzip often waits for a full buffer.

Usage and cost while streaming

Billing is the same as non-streaming: you pay for the tokens generated. If the user closes the tab mid-answer, your server should stop reading the stream; you only pay for what was produced.

Try it with one API key

Claude, GPT, Gemini, DeepSeek, image and video models through one OpenAI- and Anthropic-compatible endpoint. Prepaid, pay only for what you use.

Keep reading