Search IOCombats

Search challenges, guides, questions and articles

Your AI assistant can issue refunds. How would you design human-in-the-loop approval for risky actions?

Advanced14 min interview
Skills tested:
Classifying tools by risk and reversibilityConfiguring per-tool, input-aware approval rules on the serverRendering approval requests with clear, specific detailsSending approval responses and resuming the conversationKeeping authorisation on the server even after approval

Advertisement

🧩 Scenario

Your company wants the support assistant to handle refunds end to end. Product wants it fast, finance wants no surprise refunds, and security worries about prompt injection from customer messages. The interviewer asks you to design the approval flow, from tool definition to the button the support agent clicks.

Architecture Walkthrough

Start with a risk table, not with code

Before writing anything, list each tool and ask two questions: can it change something outside the app, and can it be undone easily? A rough split:

  • Read only (lookups, search): run automatically.
  • Reversible writes (draft an email, add a note): often automatic, sometimes approval.
  • Irreversible or costly (refunds, sending messages, deleting data): approval required, or approval above a threshold.

This is the same thinking as a confirm dialog before "Delete repository" on GitHub. The difference is that the model, not the user, is proposing the action.

Where the rule lives

The approval decision must live on the server, in code you control. In AI SDK 7 you configure it with the toolApproval option on streamText, generateText or ToolLoopAgent. For each tool you return a status: 'not-applicable', 'approved', 'denied' or 'user-approval'. A function can look at the actual input, so "refunds above a limit need approval" is one line. An older tool-level needsApproval property still exists but is marked deprecated.

The two-request flow

  1. The model calls issueRefund. The rule returns 'user-approval'.
  2. The call does not pause. The response ends with the tool part in state approval-requested, carrying an approval.id.
  3. The UI shows a card with the real details and Approve and Deny buttons.
  4. Clicking calls addToolApprovalResponse({ id, approved, reason }). With sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses, useChat resends the conversation.
  5. The next server request either runs the tool or records the denial. On denial, the model sees that the call was refused and can explain or suggest something else.

Approval is not authorisation

An approval says "a human agreed to this action". It does not prove that the human is allowed to approve it. execute must still check the session, the order owner, the refund limit for that role and idempotency (so a double click cannot refund twice). The approval response also comes from the browser, so the AI SDK offers experimental_toolApprovalSecret to sign approval requests on the server and reject forged ones.

Make the card worth reading

Show the amount, the order, the customer and the model's stated reason. If every card says "Allow action?", people stop reading.


Key Code Explained

// app/api/support/route.ts
import {
  streamText, tool, isStepCount, convertToModelMessages,
  toUIMessageStream, createUIMessageStreamResponse, type UIMessage,
} from 'ai';
import { z } from 'zod';

const AUTO_REFUND_LIMIT = 500; // example, set by finance

const tools = {
  getOrder: tool({
    description: 'Look up an order by id.',
    inputSchema: z.object({ orderId: z.string() }),
    execute: async ({ orderId }) => db.orders.find(orderId), // your data layer
  }),
  issueRefund: tool({
    description: 'Refund part or all of an order.',
    inputSchema: z.object({
      orderId: z.string(),
      amount: z.number().positive(),
      reason: z.string().max(200),
    }),
    execute: async ({ orderId, amount }, { toolCallId }) => {
      // Approval happened, but authorisation still runs here
      await assertCanRefund(currentUser(), orderId, amount);
      return payments.refund({ orderId, amount, idempotencyKey: toolCallId });
    },
  }),
};

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: 'anthropic/claude-sonnet-5.5', // example id
    messages: await convertToModelMessages(messages),
    tools,
    stopWhen: isStepCount(5),
    toolApproval: {
      getOrder: 'not-applicable', // safe read
      issueRefund: ({ amount }) =>
        amount <= AUTO_REFUND_LIMIT
          ? 'approved'
          : { type: 'user-approval', reason: `Refund of ${amount} is above the auto limit` },
    },
    experimental_toolApprovalSecret: process.env.TOOL_APPROVAL_SECRET,
  });

  return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }) });
}
'use client';
import { useChat } from '@ai-sdk/react';
import { lastAssistantMessageIsCompleteWithApprovalResponses } from 'ai';

export function SupportChat() {
  const { messages, addToolApprovalResponse } = useChat({
    // Resend automatically once every pending approval has an answer
    sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses,
  });

  return messages.flatMap((m) =>
    m.parts.map((part, i) => {
      if (part.type === 'tool-issueRefund' && part.state === 'approval-requested') {
        const { orderId, amount, reason } = part.input as { orderId: string; amount: number; reason: string };
        return (
          <div key={part.toolCallId} role="group" aria-label="Refund approval">
            <p>Refund {amount} on order {orderId}</p>
            <p>Model reason: {reason}</p>
            {part.approval.requestReason && <p>{part.approval.requestReason}</p>}
            <button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: true })}>Approve</button>
            <button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: false, reason: 'Declined by agent' })}>Deny</button>
          </div>
        );
      }
      return part.type === 'text' ? <p key={i}>{part.text}</p> : null;
    }),
  );
}

The reason on the user-approval status is attached to the approval request, so the card can show why approval was needed. Using toolCallId as the idempotency key means a replayed request cannot refund twice.


Tradeoffs

DesignProsCons
Approve every toolMaximum controlApproval fatigue, slow UX
Threshold rules on inputFast for small, safe actionsThresholds need tuning and review
Prompt says "ask first"Zero codeNot enforced, injection can bypass it
Approval plus server authorisationDefence in depthMore code paths to test
Signed approvalsBlocks forged approvalsExtra secret to manage, option is experimental

What Interviewers Actually Check

  • Whether the approval rule is enforced in server code, not in the prompt
  • Whether you understand the call returns an approval request rather than blocking
  • Whether the UI shows specific, readable details
  • Whether you still authorise inside execute and handle idempotency
  • Whether you balance safety with approval fatigue

Follow-Up Questions

  1. How would you route approvals to a different person, such as a team lead, instead of the chat user?
  2. What should happen if the user never answers an approval request?
  3. How would you log approvals and denials for an audit trail?
  4. How would you let the user edit the amount before approving?
  5. How does prompt injection in a customer's message change your threshold design?

Common Candidate Mistakes

  • Relying on the system prompt to make the model ask for confirmation.
  • Letting the client decide which calls need approval.
  • Approving based on the tool name alone when the risk depends on the input.
  • Forgetting idempotency, so a retried request repeats a payment.
  • Not handling output-denied, so the chat looks stuck after Deny.

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

  • Can you sort tools into automatic, threshold and always-approve groups?
  • Can you write a toolApproval rule based on input?
  • Can you render an approval card and send addToolApprovalResponse?
  • Can you explain the two requests involved and what the model sees on denial?
  • Can you explain why approval does not replace authorisation?

Summary

Human-in-the-loop approval starts with a risk table: reads run automatically, risky or irreversible actions wait for a person. In AI SDK 7, the server decides through toolApproval, which can return 'user-approval' based on the actual input. The call finishes with an approval request, the UI shows a specific card, the user's decision goes back through addToolApprovalResponse, and the next request runs or denies the tool. Keep the rule on the server, keep authorisation and idempotency inside execute, sign approvals for sensitive tools, and ask for approval only where it matters so people keep reading the cards.

Frequently Asked Questions

Does the server keep the request open while it waits for the user?

No. In the AI SDK the call does not pause. It finishes and returns an approval request part. When the user decides, the client sends the conversation again with the approval response, and a new request runs or denies the tool.

Can a user fake an approval from the browser?

The approval response comes from the client, so treat it as user input. Keep authorisation checks inside execute, and consider the AI SDK option experimental_toolApprovalSecret, which signs approval requests on the server and verifies them when they come back.

Should every tool need approval?

No. If everything needs a click, users approve without reading. Reserve approval for actions that move money, send messages, delete data or are hard to undo, and let safe reads run automatically.

Advertisement


Stay Updated

Get the latest frontend challenges, interview questions and tutorials delivered to your inbox.

Namaste Dev
Sponsored

Cohort-style courses to go from tutorials to job-ready.

Explore Courses

Advertisement