Ai Interview Questions

The context window is the maximum number of tokens a model can work with in one request: system prompt, chat history, documents, tools and the answer it generates. The model has no memory between calls, so your app resends the history every turn and it keeps growing. When the input is too large the API rejects the request, so the app must trim, summarize or compact older turns before that happens.

Input tokens are everything you send; output tokens are what the model generates. Output is priced higher per token (Anthropic currently lists it at 5x input for its Claude models), and it is generated one token at a time, so long answers dominate total response time. Long inputs cost more and delay the first token, but caching repeated prefixes can make that input much cheaper.

LLMs sample each next token from a probability distribution, so two runs of the same prompt can take different paths, and even temperature 0 is not fully deterministic. Model updates and changes to the hidden context (system prompt, history, retrieved data) change answers too. The UI should treat every response as a unique record: store what the user saw, offer regenerate instead of hiding variation, and validate anything the app depends on.

At each step the model produces a probability for every possible next token, and temperature reshapes that distribution before one token is sampled. Lower temperature makes the likeliest tokens even more likely, so output is more focused and repeatable; higher temperature flattens it, so output is more varied. Lower it for extraction, classification and code, but check your model first: several newer models only accept the default value.

A token is the unit of text a model reads and writes: usually a word, part of a word, a punctuation mark or a space-plus-word chunk. Providers bill and rate-limit by tokens, not characters or words, so the same text can cost different amounts on models with different tokenizers. Count tokens with the provider's tokenizer or counting API instead of guessing from string length.

Create an AbortController per request, pass controller.signal to fetch, and call controller.abort() from the Stop button or a useEffect cleanup. The fetch promise or the pending reader.read() then rejects with an AbortError, which you catch and treat as a normal stop, keeping the text already shown. On the server, pass the incoming request signal on to the upstream LLM call so the model actually stops generating and billing.

Start the upstream request, then hand its body stream straight to your response instead of awaiting text() or json(). In a Next.js Route Handler that means returning new Response(upstream.body); in Express, pipe Readable.fromWeb(upstream.body) into res. Then remove hidden buffers along the path (compression middleware, nginx proxy buffering, caches) and abort the upstream call when the client disconnects.

response.body is a ReadableStream of Uint8Array chunks. Get a reader with getReader(), call read() in a loop until done is true, and turn each chunk into text with a single TextDecoder using decode(chunk, { stream: true }). Then buffer the text and parse only complete lines or events, because network chunks do not line up with tokens or messages.

UTF-8 encodes many characters in 2 to 4 bytes, and the network can split a stream anywhere, including in the middle of one character. If you decode each chunk on its own, the incomplete bytes become the replacement character U+FFFD. Use one TextDecoder for the whole stream with decode(chunk, { stream: true }) (or TextDecoderStream), and flush it at the end, so incomplete bytes wait for the next chunk.

For a normal chat completion, Server-Sent Events (SSE) over HTTP is the better fit: the client sends one request, the server pushes tokens back in one direction, and it works with ordinary HTTP infrastructure, auth and serverless functions. The major LLM APIs stream over SSE for exactly this reason. WebSockets make sense when both sides need to keep sending data over one long-lived connection, such as realtime voice or a persistent agent session.