Streaming an LLM response, in 4 GIFs

1. Why Streaming exists

To enable the streaming option we would need to make one change in the post request that is a single field "stream": true and it will change the response experience.

Here are the pointers we take from the gif.

1. The left side shows no streaming as the cursor blinks for 4 seconds then the whole response lands at once.
2. The right side shows the streaming where the first word shows up in about 300 milliseconds. Words flow in as the model generates them.

Both the sides have same model, same prompt, same total time it is just the right side started giving response almost 4 seconds earlier. The 4 seconds wait time for a full reply feels broken. A streamed reply that finishes in four seconds feels fast. Streaming doesn’t make the model faster it makes the wait disappear.

2. What’s on the wire

When you set stream: true, the API stops sending a single JSON blob. It opens a persistent HTTP connection and pushes events down the line as the model generates them. The format is Server-Sent Events (SSE) a web standard. Any SSE debugger will read this stream.

Here’s what comes through:

A few things to notice:

The text lives in delta.text, nested inside content_block_delta events. Those are the events we should look after.

stop_reason moved. In post 1, we saw it right there in the response JSON. Here, it arrives at the very end inside a message_delta event, just before message_stop. If the loop bails out as soon as the text stops arriving we will never see it.

Chunks don’t line up with tokens or words. You might get "Hello" in one chunk and " world" in the next, or both in one. The network decides where the cuts happens and it is not the model, not the API.

That’s what the SDK has been hiding from you.

3. Reading the stream

Streaming sounds complicated until we write the loop. It’s just reading bytes, buffering them, splitting on blank lines, and parsing JSON.

Here’s the flow:

1. The response body is a ReadableStream which can be iterated with for await.
2. Each iteration gives us bytes which we can decode to string.
3. Buffer the string. A chunk might end mid-message.
4. Split the buffer on nn — that’s the SSE message separator.
5. Keep the last item in the buffer. It might be incomplete.
6. For each complete message, find the data: line, strip the prefix, and parse the JSON.
7. If the type is content_block_delta, print delta.text.
8. If it’s message_delta, you’ve got your stop_reason.

Here is the complete sample code you can use to try out:

const prompt = process.argv[2] ?? "Count to 10, slowly.";

const response = await fetch("https://api.anthropic.com/v1/messages", {
  method: "POST",
  headers: {
    "x-api-key": process.env.ANTHROPIC_API_KEY,
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
  },
  body: JSON.stringify({
    model: "claude-opus-4-5",
    max_tokens: 1024,
    stream: true,
    messages: [{ role: "user", content: prompt }],
  }),
});

const decoder = new TextDecoder();
let buffer = "";

for await (const chunk of response.body) {
  buffer += decoder.decode(chunk, { stream: true });

  const messages = buffer.split("nn");
  buffer = messages.pop() ?? "";

  for (const message of messages) {
    const dataLine = message.split("n").find(l => l.startsWith("data: "));
    if (!dataLine) continue;

    const data = JSON.parse(dataLine.slice(6));

    if (data.type === "content_block_delta" && data.delta.type === "text_delta") {
      process.stdout.write(data.delta.text);
    }

    if (data.type === "message_delta") {
      process.stderr.write(`nn[stop_reason: ${data.delta.stop_reason}]n`);
    }
  }
}

The way it is working is that when the chunk ends in the middle of a message split("nn") leaves an incomplete fragment as the last item. pop() pulls it back into the buffer so the next chunk can finish it. Without this line, every split message crashes the parser.

data.delta.type === "text_delta" this check matters because content_block_delta can carry other delta types too: input_json_delta for tool arguments, thinking_delta for extended thinking, signature_delta for verification. For now we only care about text.

You can find the full implementation here on GitHub as well.

4. Three bugs

The code above works on a good day. Here’s what breaks it on a bad one.

The ghost stream. The issue is user navigates away with the stream keeps running and tokens keep arriving with nobody to read them. In order to fix this pass an AbortController signal to fetch and call abort() when you’re done.

The fix is an AbortController:

const controller = new AbortController();
const response = await fetch(url, { signal: controller.signal, ...options });
// later, when the user navigates away:
controller.abort();

The silent truncation. The API can send an error event mid stream during overload. If the loop only handles content_block_delta, the error gets skipped and you end up with a truncated response and no exception. The fix is to handle data.type === "error" explicitly.

if (data.type === "error") {
  throw new Error(`Stream error: ${data.error.message}`);
}

The split packet. A single SSE message can arrive in two TCP packets. Without buffering, JSON.parse throws on the half. This is what buffer = messages.pop() ?? "" fixes, it holds the incomplete piece until the next chunk completes it.