Search IOCombats

Search challenges, guides, questions and articles

Your AI feature returns product cards as JSON and sometimes crashes the page. How do you validate LLM output with Zod and handle failures?

Intermediate10 min interview
Skills tested:
Treating LLM output as untrusted input at a server boundaryUsing safeParse and reading error.issuesCatching NoObjectGeneratedError and using its text, cause and usageReturning a discriminated union result the React UI can renderLogging enough to debug without leaking details to users

Advertisement

🧩 Scenario

A "suggest similar products" feature asks the model for JSON and renders cards. Once in a while price comes back as "Rs 499" instead of a number and the card component throws. The interviewer asks how you would make this safe.

Architecture Walkthrough

The model is a third-party API you do not control

You would never take a response from a payments API, cast it with as PaymentStatus and render it. Model output deserves the same care. JSON.parse(text) as Product[] gives you a type at compile time and zero protection at runtime. The "Rs 499" bug is exactly the gap between the type you declared and the data you got.

Put the check at the server boundary

The flow should be: model reply, then schema check, then your code. Do this in the route handler or server action, so nothing invalid gets saved, cached or sent to the browser. The client receives either valid cards or a clear failure, never something in between.

safeParse turns failure into data

schema.safeParse(value) returns { success: true, data } or { success: false, error }. error.issues tells you the path (['cards', 2, 'price']) and what was expected. That is far more useful in logs than "JSON was wrong".

Let the SDK do the first check

With the AI SDK, generateText with output: Output.object({ schema }) parses and validates for you. When the reply cannot be parsed or does not match, it throws NoObjectGeneratedError. Per the docs, it carries text (the raw reply), cause (the parse or validation error), response (metadata) and usage (tokens). Catch it, log it, and return a typed failure.

Give the UI a discriminated union

Return { ok: true, cards } or { ok: false, reason }. React already handles this pattern well: TypeScript narrows on ok, and each branch renders its own state.

Log for yourself, show less to the user

When a check fails, log the prompt version, the raw text, the Zod issues and token usage. That is what lets you reproduce the bug and see whether one prompt change made failures more common. The user only needs a short message and a way to try again. A failed object still cost tokens, so counting failures per feature also tells you what the bad outputs are costing you.


Key Code Explained

// app/actions/similar-products.ts
'use server';
import { generateText, Output, NoObjectGeneratedError } from 'ai';
import { z } from 'zod';
import { chatModel } from '@/lib/models';

const cardSchema = z.object({
  name: z.string().min(1),
  priceInr: z.number().positive().describe('Price in rupees, number only'),
  reason: z.string().max(140),
});
const resultSchema = z.object({ cards: z.array(cardSchema) });

export type SimilarResult =
  | { ok: true; cards: z.infer<typeof cardSchema>[] }
  | { ok: false; reason: 'invalid_output' | 'unknown' };

export async function getSimilar(productName: string): Promise<SimilarResult> {
  try {
    const { output } = await generateText({
      model: chatModel,
      output: Output.object({ schema: resultSchema }),
      prompt: `Suggest 3 products similar to: ${productName}`,
    });
    return { ok: true, cards: output.cards };
  } catch (error) {
    if (NoObjectGeneratedError.isInstance(error)) {
      // Raw text and cause go to logs, never to the user
      console.error('similar-products invalid output', {
        cause: error.cause,
        text: error.text,
        usage: error.usage,
      });
      return { ok: false, reason: 'invalid_output' };
    }
    return { ok: false, reason: 'unknown' };
  }
}

// Same idea when you get JSON from somewhere else (a queue, a cache, an old call)
export function readCachedCards(raw: unknown) {
  const parsed = resultSchema.safeParse(raw);
  if (!parsed.success) {
    console.error(parsed.error.issues); // [{ path: ['cards', 2, 'priceInr'], ... }]
    return null;
  }
  return parsed.data.cards;
}

The schema is the single source of truth: it validates at runtime, gives TypeScript the type through z.infer, and its .describe text guides the model. Keep it in one file and import it everywhere.


Tradeoffs

ApproachSafetyCost
JSON.parse plus a castNone at runtimeCrashes in production on odd output
Zod safeParse on the serverFull shape and value checkSlightly more code, one schema to maintain
Output.object in the AI SDKSame check, built inMust catch NoObjectGeneratedError yourself
Validate only in the clientProtects rendering onlyBad data may already be stored or cached

What Interviewers Actually Check

  • Whether you call LLM output untrusted, like any external API
  • Whether you validate on the server before saving or returning
  • Whether you use safeParse and read error.issues
  • Whether you give the UI a typed failure state instead of throwing
  • Whether you log the raw text for debugging and keep it away from users

Follow-Up Questions

  1. How would you retry once with the validation issues included in the prompt?
  2. Where would you keep the schema so the server and client share one type?
  3. How do you track how often validation fails per prompt version?
  4. What changes if you stream the object to the client instead of waiting?
  5. Should one bad card fail the whole list, or should you drop just that card?

Common Candidate Mistakes

  • Trusting a TypeScript cast as if it were a runtime check
  • Letting a thrown parse error reach an error boundary and blank the page
  • Showing raw model text as a fallback, which can be confusing or unsafe
  • Logging nothing, so the failure can never be reproduced
  • Writing separate TypeScript types and Zod schemas that drift apart

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

  • Can you explain why as Product[] gives no runtime safety?
  • Can you write a safeParse check and read the issue paths?
  • Can you catch NoObjectGeneratedError and name its properties?
  • Can you return a discriminated union and render each state in React?
  • Can you decide what goes in logs and what the user sees?

Summary

Model output is untrusted input, so validate it at the server boundary with a Zod schema before it is saved or rendered. Use safeParse to turn failure into a value and read error.issues for the exact path. With the AI SDK, Output.object validates for you and throws NoObjectGeneratedError, which carries the raw text, the cause and token usage for your logs. Return a typed { ok } result so React can render success and failure cleanly, and never show raw model text to the user.

Frequently Asked Questions

Should I validate on the client or the server?

On the server, before you save or return anything. The client can trust a typed response from your own API. Validating only in the browser means bad data may already be in your database.

parse or safeParse?

safeParse. It returns { success, data } or { success, error } instead of throwing, which makes the failure path explicit and easy to test.

What should the user see when validation fails?

A clear, generic message and a retry button. Do not show raw model text or Zod error details, they confuse users and can leak internal information.

Advertisement


Stay Updated

Get the latest frontend challenges, interview questions and tutorials delivered to your inbox.

Namaste Dev
Sponsored

Cohort-style courses to go from tutorials to job-ready.

Explore Courses

Advertisement