Concept Walkthrough

What a token actually is

An LLM never sees your string as characters. Before the model runs, a tokenizer splits the text into chunks from a fixed vocabulary and maps each chunk to an integer ID. Those IDs are the tokens. Common words are often one token, rarer words get split into pieces, and spaces and punctuation are usually folded into the chunks.

A useful frontend analogy: think of the tokenizer like a font's glyph table. You type a string, but what gets rendered is a sequence of glyph IDs. The model only works with the IDs, and it also produces its answer as a sequence of IDs that are decoded back into text.

OpenAI's help docs give a rough rule for English: about 4 characters, or about three quarters of a word, per token. Treat that as a ballpark, not a formula. Code, JSON, URLs and non-English scripts can tokenize very differently.

Why tokens drive cost

Providers measure and bill usage in tokens because that is what the model processes. Anthropic's API reference states it directly: the API bills and rate-limits by token counts. This has three practical effects:

1. Price: you pay per input token and per output token, usually quoted per million tokens.
2. Limits: context windows and maximum output lengths are measured in tokens, not characters.
3. Rate limits: tokens-per-minute limits are counted in tokens too.

Why the tokenizer matters

Each model family ships its own tokenizer. OpenAI's GPT-4o family uses the o200k_base encoding, older GPT-4 models use cl100k_base, and Claude models use Anthropic's own tokenizer. Anthropic documents that Claude Opus 4.7 and later produce more tokens for the same text than earlier Claude models. So "same prompt, new model" can mean a different bill even if the price per token did not change.

Everything in the request counts

Input tokens are not just the user's latest message. The system prompt, the full chat history you resend, tool definitions and any documents or images all count. In a chat UI, the history grows every turn, so input cost grows with conversation length.

Key Code Explained

// 1. Local estimate with js-tiktoken (only accurate for OpenAI models using this encoding)
import { getEncoding } from 'js-tiktoken';

const enc = getEncoding('o200k_base');

export function estimateTokens(text: string): number {
  return enc.encode(text).length;
}

console.log(estimateTokens('Explain the React reconciliation algorithm.'));

// 2. Exact pre-flight count from the provider (Anthropic TypeScript SDK)
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic(); // reads ANTHROPIC_API_KEY on the server

export async function countPromptTokens(model: string, userText: string) {
  const res = await client.messages.countTokens({
    model,
    system: 'You are a helpful frontend interview coach.',
    messages: [{ role: 'user', content: userText }],
  });
  return res.input_tokens; // system prompt + messages, as the model will see them
}

// 3. Real usage after the call: this is what you actually pay for
export async function ask(model: string, userText: string) {
  const msg = await client.messages.create({
    model,
    max_tokens: 512,
    messages: [{ role: 'user', content: userText }],
  });
  const { input_tokens, output_tokens } = msg.usage;
  return { msg, input_tokens, output_tokens };
}

Three levels of accuracy: the local tokenizer is free and instant but only an estimate (and only for matching models). The provider's counting endpoint matches the model's tokenizer; Anthropic notes it is an estimate that can differ slightly from the final count. The usage object on the response is the number you are billed on, so log that.

Tradeoffs

What Interviewers Actually Check

• Whether you can define a token without saying "a word"
• Whether you connect tokens to price, context window size and rate limits
• Whether you know tokenizers differ across model families and versions
• Whether you count everything in the request: system prompt, history, tools
• Whether you know where the real billed number comes from (the response usage)

Follow-Up Questions

1. Why might a JSON payload use more tokens than the same data written as plain sentences?
2. How would you cap the cost of a single chat request in your API route?
3. Where would you store per-request token usage so you can report cost per feature?
4. Why does resending chat history every turn make long conversations more expensive?
5. Should the browser ever call the provider's counting API directly? Why not?

Common Candidate Mistakes

• Saying a token is a word, or a character
• Using string.length and presenting the result as an exact cost
• Using an OpenAI tokenizer to count tokens for a Claude or Gemini model and trusting the number
• Forgetting that the system prompt and earlier messages are billed on every request
• Assuming that a model upgrade at the same price per token keeps the same cost

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

• Can you explain what a tokenizer does in two sentences?
• Can you list three things tokens determine (price, context limit, rate limits)?
• Can you count tokens locally and with a provider API in TypeScript?
• Can you explain why the same prompt costs differently on two models?
• Can you point to the field in a response that tells you what you were billed for?

Summary

A token is a chunk of text from the model's vocabulary, mapped to an integer ID. Models read and write tokens, so providers bill, limit and rate-limit in tokens. For English, roughly 4 characters per token is a fair estimate, but tokenizers differ between model families and even between versions of the same family, so the same text can cost different amounts. In TypeScript, estimate with a local tokenizer like js-tiktoken for matching models, use the provider's counting API for pre-flight checks, and treat the usage object on each response as the source of truth for cost.