LLMs for Frontend Engineers: Tokens, Context Windows and Cost Explained

You ship a small AI chat feature, it works perfectly in the demo, and two weeks later the bill is far higher than you expected, or long conversations start failing with "prompt is too long". Both problems come from the same three ideas: tokens, the context window, and the fact that the model remembers nothing between requests. This post explains LLM tokens and the context window from a frontend engineer's point of view, with TypeScript you can run today.

By the end you will be able to estimate the size of any request, read the usage numbers an API returns, and keep a chat inside its token budget.

Tokens: the unit everything is measured in

An LLM does not read characters or words. Before your text reaches the model, a tokenizer splits it into tokens: chunks that are often a whole short word, part of a longer word, a space plus a word, or punctuation. Each token maps to an integer ID, and the model only ever sees that array of integers.

Think of it like a bundler. You write readable source, but what you ship and pay for is the bundle size in bytes. With an LLM you write prompts in natural language, but what gets limited and billed is the token count.

OpenAI's own rule of thumb for English is that one token is roughly 4 characters, or about three quarters of a word. That is only an estimate, and it breaks down quickly for code, JSON and non-English text. You can check this yourself with js-tiktoken, a pure JavaScript port of OpenAI's tokenizer:

// npm i js-tiktoken   (run with: npx tsx tokens.ts)
import { getEncoding } from 'js-tiktoken';

const enc = getEncoding('o200k_base'); // encoding used by recent OpenAI models

const order = {
  orderId: 'ORD-1042',
  items: [{ sku: 'TSHIRT-M', qty: 2 }],
  status: 'shipped',
};

const samples = {
  english: 'Please summarise this order for the customer.',
  hindi: 'कृपया ग्राहक के लिए इस ऑर्डर का सारांश दें।',
  json: JSON.stringify(order),
  prettyJson: JSON.stringify(order, null, 2),
};

for (const [label, text] of Object.entries(samples)) {
  console.log(
    label,
    text.length,
    'chars ->',
    enc.encode(text).length,
    'tokens',
  );
}

// See how a word is split
console.log(enc.encode('useEffect').map((id) => enc.decode([id])));
// [ 'use', 'Effect' ]

When I ran a version of this with o200k_base, the English sentence was 9 tokens, while the Hindi sentence with the same meaning and a similar character count was 16 tokens. A small JSON object went from 29 tokens minified to 49 tokens when pretty-printed with two-space indentation. Two practical lessons follow: Indic-language prompts can cost noticeably more for the same meaning, and formatting whitespace in data you send to the model is not free.

One caution: every model family has its own tokenizer. js-tiktoken is accurate for OpenAI encodings and only an approximation for other providers. For exact numbers, use the usage data the API returns (shown below) or the provider's token counting endpoint.

The model is a pure function with no memory

This is the idea that surprises most frontend engineers. An LLM API call behaves like a pure function:

type Llm = (messages: Message[]) => string;

It has no useState. It does not remember your previous request. A chat "remembers" earlier turns only because your code sends the full conversation again on every call, the same way a React component only knows what is in its current props.

So when a user sends their tenth message, the request contains the system instructions, all nine earlier user messages, all nine earlier assistant replies, and the new message. OpenAI's conversation state guide is explicit that even when you chain turns with previous_response_id, all earlier input tokens in the chain are still billed as input tokens. Server-side state saves you bandwidth, not tokens.

Diagram
100%
sequenceDiagram
    participant UI as React chat UI
    participant API as Your Node route
    participant LLM as Model
    UI->>API: turn 1: [user1]
    API->>LLM: [instructions, user1]
    LLM-->>API: reply1
    UI->>API: turn 2: [user1, reply1, user2]
    API->>LLM: [instructions, user1, reply1, user2]
    LLM-->>API: reply2
    Note over API,LLM: Input grows every turn because the whole history is resent
Zoom or open fullscreen for a closer look
visualized by

The context window: one budget for input and output

The context window is the maximum number of tokens a model can handle in a single request. The important detail is what counts against it. Both OpenAI and Anthropic document that the window covers input and output together: instructions, every message, tool definitions, documents, the generated reply and any reasoning tokens.

A good analogy is a fixed-height viewport. Everything has to fit inside it at once. If your history takes most of the space, there is less room left for the answer.

Diagram
100%
flowchart LR
    A["Instructions"] --> W["Context window (fixed per model)"]
    B["Chat history"] --> W
    C["Tool definitions and documents"] --> W
    D["New user message"] --> W
    W --> E["Room left for output and reasoning"]
Zoom or open fullscreen for a closer look
visualized by

What happens when you overflow depends on the provider and model. Anthropic's docs say an input that alone exceeds the window is rejected with a 400 error, while on newer Claude models a reply can be cut off when generation hits the limit. OpenAI's docs say tokens beyond the limit may be truncated. Either way your UI gets an incomplete answer or an error, so you need to plan for it.

Bigger is also not automatically better. Anthropic's documentation notes that accuracy and recall degrade as the token count grows, which it calls context rot. Sending less, but more relevant, context usually gives better answers and a smaller bill.

Cost: you pay per token, and turns add up

Providers price input tokens and output tokens separately, per million tokens, and output is usually priced higher than input. Always read the current numbers from the provider's pricing page; they change often.

The surprise is how fast input grows in a chat. Take an illustrative conversation (assumed sizes, not measurements): 500 tokens of instructions, and each turn adds a 150-token user message and a 350-token reply.

Input on turn n is 500 + 500 × (n − 1) + 150, so it grows linearly per turn, and the total grows roughly with the square of the conversation length. In this example, input tokens over the whole chat are about 15 times the output tokens. That is why trimming history matters more than shortening replies.

Build it: measure usage and estimate cost in TypeScript

The Vercel AI SDK (version 7, which needs Node 22+ and ESM) returns normalized usage for every provider. Here is a script that sends a prompt, caps the output, and reports usage:

// npm i ai @ai-sdk/openai   (set OPENAI_API_KEY, run with: npx tsx usage.ts)
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-5-mini'),
  instructions: 'You are a concise assistant for a React codebase.',
  prompt: 'Explain useMemo vs useCallback in three bullet points.',
  maxOutputTokens: 400,
});

const { inputTokens, outputTokens, totalTokens, inputTokenDetails } =
  result.usage;

console.log(result.text);
console.log({ inputTokens, outputTokens, totalTokens });
console.log('cached input tokens:', inputTokenDetails?.cacheReadTokens ?? 0);

if (result.finishReason === 'length') {
  console.warn('Reply hit maxOutputTokens and was cut off.');
}

Now turn usage into money. Keep prices in one config object, filled in from the pricing page, so you never hard-code them across the codebase:

// cost.ts
import type { LanguageModelUsage } from 'ai';

type Price = {
  inputPerMillion: number;
  cachedInputPerMillion: number;
  outputPerMillion: number;
};

// Fill these from your provider's current pricing page.
export const PRICES: Record<string, Price> = {
  'gpt-5-mini': {
    inputPerMillion: Number(process.env.PRICE_IN),
    cachedInputPerMillion: Number(process.env.PRICE_CACHED),
    outputPerMillion: Number(process.env.PRICE_OUT),
  },
};

const perToken = (perMillion: number) => perMillion / 1_000_000;

export function estimateCostUsd(
  model: string,
  usage: LanguageModelUsage,
): number {
  const price = PRICES[model];
  if (!price) throw new Error(`No price configured for ${model}`);

  const cached = usage.inputTokenDetails?.cacheReadTokens ?? 0;
  const uncached = (usage.inputTokens ?? 0) - cached;

  return (
    uncached * perToken(price.inputPerMillion) +
    cached * perToken(price.cachedInputPerMillion) +
    (usage.outputTokens ?? 0) * perToken(price.outputPerMillion)
  );
}

Log this next to each request in development and you will see exactly which feature is expensive.

Build it: keep a chat inside a token budget

The simplest production pattern is a sliding window: always keep the instructions, then keep the newest messages until a budget is reached. It is the same idea as list virtualization in React, where you render only the rows that fit.

// trim-history.ts
import { getEncoding } from 'js-tiktoken';

type Message = { role: 'user' | 'assistant'; content: string };

const enc = getEncoding('o200k_base');
const countTokens = (text: string) => enc.encode(text).length;

export function trimToBudget(history: Message[], budget: number): Message[] {
  const kept: Message[] = [];
  let used = 0;

  // Walk from newest to oldest, keep what fits
  for (let i = history.length - 1; i >= 0; i--) {
    const cost = countTokens(history[i].content);
    if (used + cost > budget) break;
    kept.unshift(history[i]);
    used += cost;
  }
  return kept;
}

// Usage: leave headroom for instructions and the reply
// const messages = trimToBudget(fullHistory, 6_000);

The token count here is an estimate, so pick a budget well below the model's real limit. For long conversations, a common next step is to replace the dropped turns with a short summary message instead of losing them completely.

Common mistakes and their fixes

1. Estimating with text.length / 4 everywhere. It is wrong for code, JSON and Hindi or other Indic scripts. Fix: use a tokenizer for estimates and trust result.usage for real numbers.
2. Forgetting that output shares the window. A huge prompt leaves no room for the answer. Fix: reserve space for output and set maxOutputTokens, then check finishReason === 'length' and show a "continue" option in the UI.
3. Sending the full history forever. Cost grows with every turn and quality can drop. Fix: trim to a token budget and summarise older turns.
4. Putting changing content at the start of the prompt. OpenAI's caching applies automatically to prompts over 1,024 tokens, but only when the beginning of the prompt is identical. Fix: put static instructions, examples and tool definitions first, and per-user data last.
5. Pretty-printing JSON and pasting whole API responses into prompts. Fix: send minified JSON with only the fields the model needs.

Interview angle

For AI-focused frontend roles, interviewers often ask how a chat "remembers" earlier messages, why a long conversation suddenly fails, or how you would control cost for a feature with many users. A strong answer covers statelessness, the shared input plus output budget, trimming or summarising history, and reading usage from the API. Practise these in the LLM fundamentals interview questions.

What's next

On Thursday, the case study How a chat AI handles your message, end to end follows a single message from the input box, through tokenization and the model, and back to the screen. Next week we start streaming those tokens to the browser from Node.

Conclusion

Tokens are the unit of size and cost, the context window is one fixed budget for both input and output, and the model is stateless, so every turn resends the history. Measure with a tokenizer, trust the usage numbers the API returns, keep static prompt content first, and trim history to a budget. Those four habits prevent most surprise bills and most "prompt is too long" errors.