Your AI assistant can issue refunds. How would you design human-in-the-loop approval for risky actions?
Advertisement
🧩 Scenario
Architecture Walkthrough
Start with a risk table, not with code
Before writing anything, list each tool and ask two questions: can it change something outside the app, and can it be undone easily? A rough split:
- Read only (lookups, search): run automatically.
- Reversible writes (draft an email, add a note): often automatic, sometimes approval.
- Irreversible or costly (refunds, sending messages, deleting data): approval required, or approval above a threshold.
This is the same thinking as a confirm dialog before "Delete repository" on GitHub. The difference is that the model, not the user, is proposing the action.
Where the rule lives
The approval decision must live on the server, in code you control. In AI SDK 7 you configure it with the toolApproval option on streamText, generateText or ToolLoopAgent. For each tool you return a status: 'not-applicable', 'approved', 'denied' or 'user-approval'. A function can look at the actual input, so "refunds above a limit need approval" is one line. An older tool-level needsApproval property still exists but is marked deprecated.
The two-request flow
- The model calls
issueRefund. The rule returns'user-approval'. - The call does not pause. The response ends with the tool part in state
approval-requested, carrying anapproval.id. - The UI shows a card with the real details and Approve and Deny buttons.
- Clicking calls
addToolApprovalResponse({ id, approved, reason }). WithsendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses,useChatresends the conversation. - The next server request either runs the tool or records the denial. On denial, the model sees that the call was refused and can explain or suggest something else.
Approval is not authorisation
An approval says "a human agreed to this action". It does not prove that the human is allowed to approve it. execute must still check the session, the order owner, the refund limit for that role and idempotency (so a double click cannot refund twice). The approval response also comes from the browser, so the AI SDK offers experimental_toolApprovalSecret to sign approval requests on the server and reject forged ones.
Make the card worth reading
Show the amount, the order, the customer and the model's stated reason. If every card says "Allow action?", people stop reading.
Key Code Explained
// app/api/support/route.ts
import {
streamText, tool, isStepCount, convertToModelMessages,
toUIMessageStream, createUIMessageStreamResponse, type UIMessage,
} from 'ai';
import { z } from 'zod';
const AUTO_REFUND_LIMIT = 500; // example, set by finance
const tools = {
getOrder: tool({
description: 'Look up an order by id.',
inputSchema: z.object({ orderId: z.string() }),
execute: async ({ orderId }) => db.orders.find(orderId), // your data layer
}),
issueRefund: tool({
description: 'Refund part or all of an order.',
inputSchema: z.object({
orderId: z.string(),
amount: z.number().positive(),
reason: z.string().max(200),
}),
execute: async ({ orderId, amount }, { toolCallId }) => {
// Approval happened, but authorisation still runs here
await assertCanRefund(currentUser(), orderId, amount);
return payments.refund({ orderId, amount, idempotencyKey: toolCallId });
},
}),
};
export async function POST(req: Request) {
const { messages }: { messages: UIMessage[] } = await req.json();
const result = streamText({
model: 'anthropic/claude-sonnet-5.5', // example id
messages: await convertToModelMessages(messages),
tools,
stopWhen: isStepCount(5),
toolApproval: {
getOrder: 'not-applicable', // safe read
issueRefund: ({ amount }) =>
amount <= AUTO_REFUND_LIMIT
? 'approved'
: { type: 'user-approval', reason: `Refund of ${amount} is above the auto limit` },
},
experimental_toolApprovalSecret: process.env.TOOL_APPROVAL_SECRET,
});
return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }) });
}
'use client';
import { useChat } from '@ai-sdk/react';
import { lastAssistantMessageIsCompleteWithApprovalResponses } from 'ai';
export function SupportChat() {
const { messages, addToolApprovalResponse } = useChat({
// Resend automatically once every pending approval has an answer
sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses,
});
return messages.flatMap((m) =>
m.parts.map((part, i) => {
if (part.type === 'tool-issueRefund' && part.state === 'approval-requested') {
const { orderId, amount, reason } = part.input as { orderId: string; amount: number; reason: string };
return (
<div key={part.toolCallId} role="group" aria-label="Refund approval">
<p>Refund {amount} on order {orderId}</p>
<p>Model reason: {reason}</p>
{part.approval.requestReason && <p>{part.approval.requestReason}</p>}
<button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: true })}>Approve</button>
<button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: false, reason: 'Declined by agent' })}>Deny</button>
</div>
);
}
return part.type === 'text' ? <p key={i}>{part.text}</p> : null;
}),
);
}
The reason on the user-approval status is attached to the approval request, so the card can show why approval was needed. Using toolCallId as the idempotency key means a replayed request cannot refund twice.
Tradeoffs
| Design | Pros | Cons |
|---|---|---|
| Approve every tool | Maximum control | Approval fatigue, slow UX |
| Threshold rules on input | Fast for small, safe actions | Thresholds need tuning and review |
| Prompt says "ask first" | Zero code | Not enforced, injection can bypass it |
| Approval plus server authorisation | Defence in depth | More code paths to test |
| Signed approvals | Blocks forged approvals | Extra secret to manage, option is experimental |
What Interviewers Actually Check
- Whether the approval rule is enforced in server code, not in the prompt
- Whether you understand the call returns an approval request rather than blocking
- Whether the UI shows specific, readable details
- Whether you still authorise inside
executeand handle idempotency - Whether you balance safety with approval fatigue
Follow-Up Questions
- How would you route approvals to a different person, such as a team lead, instead of the chat user?
- What should happen if the user never answers an approval request?
- How would you log approvals and denials for an audit trail?
- How would you let the user edit the amount before approving?
- How does prompt injection in a customer's message change your threshold design?
Common Candidate Mistakes
- Relying on the system prompt to make the model ask for confirmation.
- Letting the client decide which calls need approval.
- Approving based on the tool name alone when the risk depends on the input.
- Forgetting idempotency, so a retried request repeats a payment.
- Not handling
output-denied, so the chat looks stuck after Deny.
Interview Readiness Checklist
Before you leave this question, make sure you can answer:
- Can you sort tools into automatic, threshold and always-approve groups?
- Can you write a
toolApprovalrule based on input? - Can you render an approval card and send
addToolApprovalResponse? - Can you explain the two requests involved and what the model sees on denial?
- Can you explain why approval does not replace authorisation?
Summary
Human-in-the-loop approval starts with a risk table: reads run automatically, risky or irreversible actions wait for a person. In AI SDK 7, the server decides through toolApproval, which can return 'user-approval' based on the actual input. The call finishes with an approval request, the UI shows a specific card, the user's decision goes back through addToolApprovalResponse, and the next request runs or denies the tool. Keep the rule on the server, keep authorisation and idempotency inside execute, sign approvals for sensitive tools, and ask for approval only where it matters so people keep reading the cards.
Does the server keep the request open while it waits for the user?
No. In the AI SDK the call does not pause. It finishes and returns an approval request part. When the user decides, the client sends the conversation again with the approval response, and a new request runs or denies the tool.
Can a user fake an approval from the browser?
The approval response comes from the client, so treat it as user input. Keep authorisation checks inside execute, and consider the AI SDK option experimental_toolApprovalSecret, which signs approval requests on the server and verifies them when they come back.
Should every tool need approval?
No. If everything needs a click, users approve without reading. Reserve approval for actions that move money, send messages, delete data or are hard to undo, and let safe reads run automatically.
Advertisement
Stay Updated
Get the latest frontend challenges, interview questions and tutorials delivered to your inbox.
Advertisement