Building a Streaming AI Chat UI in React: The Architecture Behind Token-by-Token Rendering

Almost every product with an AI feature now streams its responses: text appears word by word instead of arriving all at once after a multi-second wait. This is not cosmetic. It is a real architectural decision that touches HTTP, the browser's streaming APIs, and React's rendering model, and it has become a fair interview question because it tests whether you understand what happens between a server sending bytes and a user seeing words, not just whether you can call an AI SDK.

This post builds a streaming AI chat UI from first principles: how the server keeps a connection open and pushes chunks, how the browser reads those chunks before the response finishes, and how to render them in React without breaking markdown mid-token, blocking the main thread, or leaking a connection every time a user navigates away.

Why Streaming Requires a Different Transport Model

A normal fetch() call waits for the entire response body before your code gets to touch it. That works for a JSON API response but is unusable for an LLM, which can take 10 to 30 seconds to generate a full answer. Making a user stare at a blank screen for that long, when the model already knows the first word, is a bad experience for no technical reason.

The fix is to not wait. HTTP supports chunked transfer encoding, where the server sends the response in a sequence of chunks without declaring the total length up front, and closes the stream when it is done. The server writes a chunk the moment it has one (a token, or a small group of tokens, from the model), and the connection stays open until the model finishes generating.

Diagram
100%
sequenceDiagram
  participant Browser
  participant Route as Next.js Route Handler
  participant LLM as Model Provider

  Browser->>Route: POST /api/chat (fetch, streaming)
  Route->>LLM: Request completion (stream: true)
  LLM-->>Route: chunk 1 (token)
  Route-->>Browser: chunk 1 (SSE data: line)
  LLM-->>Route: chunk 2 (token)
  Route-->>Browser: chunk 2 (SSE data: line)
  Note over Browser: UI updates after each chunk, not at the end
  LLM-->>Route: [DONE]
  Route-->>Browser: stream closed
Zoom or open fullscreen for a closer look
visualized by

On the server side, a Next.js Route Handler that proxies a streaming model call looks like this. The key detail is that the handler returns a ReadableStream as the response body instead of a finished string:

// src/app/api/chat/route.ts
export async function POST(req: Request) {
  const { messages } = await req.json();

  const modelStream = await getModelCompletionStream(messages); // async iterable of tokens

  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      for await (const token of modelStream) {
        const event = `data: ${JSON.stringify({ token })}\n\n`;
        controller.enqueue(encoder.encode(event));
      }
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

The text/event-stream content type marks this as Server-Sent Events (SSE), a text-based streaming format where each message is a data: line (or set of lines) terminated by a blank line. SSE is a one-way, server-to-client channel built directly on top of HTTP, which makes it a natural fit for streaming a model's output: the browser only needs to receive, not negotiate a second connection.

Reading the Stream in the Browser

The browser's EventSource API can consume SSE directly, but it has a real limitation for AI chat: it cannot send custom headers, which means no Authorization: Bearer <token> on the initial request. Most production chat UIs instead use fetch() with its streaming response body, which supports full control over headers and the request itself while still consuming an SSE-formatted stream manually.

fetch() exposes the response body as a ReadableStream of raw bytes via response.body. Calling .getReader() gives you a reader you can pull chunks from with .read(), in a loop, as they arrive, well before the server has finished sending. Because a single UTF-8 character can be split across two chunk boundaries, decoding must go through TextDecoder with its stream: true option, which buffers any incomplete byte sequence until the next chunk completes it.

Diagram
100%
flowchart LR
  A["response.body\n(ReadableStream of bytes)"] --> B["reader.read()\n(pull loop)"]
  B --> C["TextDecoder.decode(chunk, { stream: true })"]
  C --> D["Append to text buffer"]
  D --> E{"Buffer contains\ncomplete SSE event?\n(ends in blank line)"}
  E -- No --> B
  E -- Yes --> F["Parse 'data:' line as JSON"]
  F --> G["Dispatch token to React state"]
  G --> B
Zoom or open fullscreen for a closer look
visualized by

Putting that into a working client function:

async function streamChatResponse(
  messages: ChatMessage[],
  onToken: (token: string) => void,
  signal: AbortSignal,
) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
    signal,
  });

  if (!response.body) throw new Error('No response body to stream');

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const events = buffer.split('\n\n');
    buffer = events.pop() ?? ''; // keep the last, possibly incomplete event

    for (const event of events) {
      const line = event.trim();
      if (!line.startsWith('data:')) continue;

      const payload = line.slice(5).trim();
      if (payload === '[DONE]') return;

      const { token } = JSON.parse(payload);
      onToken(token);
    }
  }
}

This is, in simplified form, the same shape used by production tools. The Vercel AI SDK's useChat hook, for example, standardized on SSE for its UI Message Stream Protocol, where the server sends typed parts (a start event, a sequence of text deltas, an end event, plus structured parts for tool calls) rather than raw tokens, so the client can reconstruct not just text but tool invocations and their results as they stream in. The mechanics underneath, a ReadableStream, a decoder, and an event parser, are exactly what you just built by hand.

Rendering Tokens Without Breaking Markdown or the Main Thread

Two problems show up the moment you try to render streamed tokens directly.

Problem one: markdown arrives sliced mid-syntax. A model might stream **bo, then ld text** a moment later. If you render on every token with a naive markdown parser, the UI flashes an unclosed bold marker, a broken code fence, or a half-written list item, then snaps to correct formatting once the stream finishes. That reads as broken software even though nothing is actually wrong.

The fix used by streaming-aware markdown renderers is block-level parsing with memoization. Instead of re-parsing the entire message on every token, the renderer splits the accumulated text into blocks (a paragraph, a code fence, a list) using blank lines and fence markers as boundaries. Every block except the last one is treated as closed and wrapped in React.memo, so React skips re-rendering it entirely on subsequent updates, its props have not changed. Only the last, still-growing block re-renders as new tokens land in it.

Diagram
100%
flowchart TD
  A["Accumulated markdown text"] --> B["Split into blocks\non blank lines / fence boundaries"]
  B --> C["Block 1: closed"]
  B --> D["Block 2: closed"]
  B --> E["Block 3: still streaming"]
  C --> F["React.memo\nprops unchanged, skip re-render"]
  D --> F
  E --> G["Re-render on every token"]
Zoom or open fullscreen for a closer look
visualized by

A simplified version of that memoized block component:

const MarkdownBlock = React.memo(function MarkdownBlock({
  content,
}: {
  content: string;
}) {
  return <ReactMarkdown>{content}</ReactMarkdown>;
});

function StreamingMessage({ text }: { text: string }) {
  const blocks = splitIntoBlocks(text); // your own boundary logic, or a library like Streamdown

  return (
    <>
      {blocks.map((block, i) => (
        <MarkdownBlock key={i} content={block} />
      ))}
    </>
  );
}

In a typical response, most of the content is in completed blocks by the time streaming finishes, so this cuts the amount of re-parsing and re-rendering work down to whatever is in the single active block, instead of the whole message, on every token.

Problem two: high-frequency state updates can block typing. If a user types in an input field while dozens of tokens per second are flowing into a setState call, React by default treats both updates as equally urgent, and the input can visibly lag behind keystrokes. React's useTransition hook solves this by letting you mark the token updates as a low-priority transition, so React finishes rendering an interruptible transition update, then yields to anything urgent, like a keystroke, before continuing.

function useStreamingMessage() {
  const [text, setText] = useState('');
  const [isPending, startTransition] = useTransition();

  const appendToken = useCallback((token: string) => {
    startTransition(() => {
      setText((prev) => prev + token);
    });
  }, []);

  return { text, isPending, appendToken };
}

This does not make the streaming render faster. It changes its priority relative to user input, which is what actually matters for perceived responsiveness: a chat box that keeps typing snappy while the assistant's message fills in underneath it, rather than freezing the whole page for a few milliseconds on every token.

Cancellation: Closing the Stream Cleanly

An open ReadableStream is a live connection. If a user navigates away, unmounts the chat panel, or clicks a "Stop generating" button, the fetch and the underlying model call need to actually stop, not keep consuming server resources and tokens in the background. AbortController is the mechanism for this, and it should be wired to every exit path, not just one:

function useChatStream() {
  const [text, setText] = useState('');
  const controllerRef = useRef<AbortController | null>(null);

  const send = useCallback(async (messages: ChatMessage[]) => {
    controllerRef.current?.abort(); // cancel any in-flight stream first
    const controller = new AbortController();
    controllerRef.current = controller;

    setText('');
    try {
      await streamChatResponse(
        messages,
        (token) => setText((prev) => prev + token),
        controller.signal,
      );
    } catch (err) {
      if ((err as Error).name !== 'AbortError') throw err;
    }
  }, []);

  const stop = useCallback(() => controllerRef.current?.abort(), []);

  useEffect(() => stop, [stop]); // abort on unmount too

  return { text, send, stop };
}

Tying the same controller to unmount, an explicit stop button, and a new message replacing an in-flight one means there is exactly one cancellation code path instead of three separate ones that can drift out of sync.

Practical Takeaway

If you are building or debugging a streaming AI feature, the questions to ask in order are: is the server actually flushing chunks as they are produced, or buffering the whole response first; is the client reading response.body incrementally or waiting on .json(); is the renderer re-parsing the entire message on every token, or only the active block; and is any state update competing with user input for priority. Each of these is a separate, checkable layer, and a slow or janky streaming UI is almost always a problem in one specific layer, not "AI is slow."

In an interview setting, being able to draw this pipeline, HTTP chunked response, SSE framing, ReadableStream and TextDecoder, block-level memoized rendering, useTransition for priority, is a stronger signal than knowing which npm package implements it, because it is the same pipeline underneath every chat UI regardless of which SDK sits on top.

Conclusion

Streaming text is not a single feature, it is a chain of decisions: how the server frames chunks, how the browser reads bytes before the response ends, and how React decides what to re-render and when. Understanding each link means you can build a streaming chat UI without an SDK, debug one that came from an SDK, and answer exactly why it behaves the way it does under load.