Architecture Walkthrough

One policy does not fit every failure

"Retry three times" sounds safe, but it treats very different problems the same way. Think of it like HTTP status codes: you retry a 503, you do not retry a 400. Sort model failures the same way.

1. Transient: retry with backoff

Timeouts, 429 rate limits and 5xx errors say nothing about your prompt. The same request can work a moment later. The AI SDK's maxRetries option (documented default 2) covers the provider call. Keep it small in user-facing routes, because each retry adds wait time.

2. Truncated: change the request

If finishReason is length, the model hit the output token limit and the JSON is likely incomplete. Anthropic and OpenAI both warn that output cut off this way may not match your schema. Retrying the same request usually gives the same result. Raise the limit, ask for less, or split the work (for example parse experience and education in separate calls).

3. Refused or filtered: fall back

A content-filter finish reason or a refusal is a decision, not a glitch. Anthropic notes a refusal still returns 200 and you are billed for the tokens. Retrying the same input mostly wastes money. Show a clear message and offer the manual path.

4. Invalid shape: repair

The JSON arrived but yearsOfExperience is "5+". Two kinds of repair, in this order:

• Repair in code. If the fix is predictable, do it with z.preprocess or a small function. It is free, instant and deterministic.
• Re-ask once. If the content is wrong in a way code cannot fix, send one more request with the previous output and the Zod issues (skills: Expected array, received string). Telling the model exactly what is wrong gives it a concrete target. Cap it at one attempt.

5. Fallbacks, from cheapest to most expensive

A partial result with empty fields highlighted, a cached or default answer, a different model, or the plain non-AI form. For a resume parser, the honest fallback is "We could not read some fields, please fill them in". The user is never stuck.

Budget everything

Each retry costs tokens and seconds. Put a total cap per request (for example one repair, provider retries only for transient errors) and log which path was taken, so you can see which failures are common.

Key Code Explained

import { generateText, Output, NoObjectGeneratedError } from 'ai';
import { z } from 'zod';
import { chatModel } from '@/lib/models';

// Code repair: "5+" or "5 years" becomes 5
const years = z.preprocess((v) => {
  if (typeof v === 'string') {
    const n = parseFloat(v);
    return Number.isNaN(n) ? v : n;
  }
  return v;
}, z.number().nonnegative());

const profileSchema = z.object({
  fullName: z.string(),
  yearsOfExperience: years.nullable(),
  skills: z.array(z.string()),
});

type ParseResult =
  | { kind: 'ok'; profile: z.infer<typeof profileSchema> }
  | { kind: 'truncated' | 'blocked' | 'invalid' };

export async function parseResume(resumeText: string): Promise<ParseResult> {
  const basePrompt = `Extract the profile from this resume:\n\n${resumeText}`;
  let prompt = basePrompt;

  for (let attempt = 0; attempt < 2; attempt++) {
    try {
      const result = await generateText({
        model: chatModel,
        output: Output.object({ schema: profileSchema }),
        maxRetries: 2, // provider call retries
        maxOutputTokens: 1500,
        prompt,
      });

      // Retrying these with the same request will not help
      if (result.finishReason === 'length') return { kind: 'truncated' };
      if (result.finishReason === 'content-filter') return { kind: 'blocked' };

      // Reading output parses and validates; it throws on a bad object
      return { kind: 'ok', profile: result.output };
    } catch (error) {
      if (!NoObjectGeneratedError.isInstance(error)) throw error;
      // One repair attempt: show the model its answer and what was wrong
      prompt = buildRepairPrompt(basePrompt, error.text ?? '');
    }
  }
  return { kind: 'invalid' }; // UI shows the manual form
}

function buildRepairPrompt(original: string, badText: string) {
  let issues = 'The previous answer was not valid JSON.';
  try {
    const check = profileSchema.safeParse(JSON.parse(badText));
    if (!check.success) {
      issues = check.error.issues
        .map((i) => `${i.path.join('.')}: ${i.message}`)
        .join('\n');
    }
  } catch {
    // keep the generic message
  }
  return `${original}\n\nYour previous answer:\n${badText}\n\nFix these problems:\n${issues}`;
}

The loop allows exactly one repair. Truncation and content filtering return straight away, because a second identical call would not help. The repair prompt reuses the same Zod schema, so the issues the model sees are the same ones your validation reports.

Tradeoffs

What Interviewers Actually Check

• Whether you classify failures before choosing a strategy
• Whether you check finishReason and treat truncation differently
• Whether you prefer deterministic code fixes over another model call
• Whether a re-ask includes the actual validation errors
• Whether attempts are capped and side effects are idempotent

Follow-Up Questions

1. How would you add exponential backoff with jitter for 429s in your own fetch layer?
2. When would you fall back to a different model, and what must you test first?
3. How do you show partial data to the user when only some fields failed?
4. What metrics would tell you that your repair rate is too high?
5. How would this change if the response streams to the UI?

Common Candidate Mistakes

• An unbounded retry loop that burns tokens on a refusal
• Retrying a cut-off response without changing the token limit or the request
• Re-asking with "try again" and no error details
• Hiding all failures behind a generic error with no manual path
• Forgetting that each retry also adds seconds for the user

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

• Can you sort failures into transient, truncated, blocked and invalid?
• Can you say which ones to retry, repair or fall back on, and why?
• Can you write a code repair with z.preprocess?
• Can you build a repair prompt from Zod issues?
• Can you explain how you cap cost and protect side effects?

Summary

Choose the strategy by the failure. Retry transient provider errors with a small maxRetries and backoff. Change the request when finishReason is length, because the same call will be cut off again. Fall back on refusals and content filters instead of retrying them. For invalid shapes, repair in code first and re-ask the model at most once with the exact Zod issues. Cap every path, keep side effects idempotent, and always give the user a manual way forward.