Customer Support Resolution Agent

What this scenario tests

This is the scenario that most directly asks where judgement belongs. The agent has a tool that moves real money, so the interesting questions are never about whether Claude can call process_refund — they are about what has to be true before it does, and what happens when the customer's request does not match any policy the agent knows about. Expect questions that describe an ambiguous request and offer four plausible handlings, only one of which respects both the stated customer preference and the policy gap.

Traps it is built to catch

• ▸Treating escalation as a confidence threshold. An agent that escalates whenever it feels unsure escalates constantly; escalation criteria are decided in advance and are about policy gaps and irreversible actions, not about a number.
• ▸Putting the guardrail in the prompt. A refund ceiling enforced by asking nicely is not enforced. Orchestration-layer checks and tool-level constraints survive a model that has been talked around.
• ▸Consolidating the tools. get_customer and lookup_order look similar enough to merge until the agent starts calling the wrong one — this scenario is where tool-description quality stops being an abstraction.