Resolve AI Agent — Autonomous Customer Operations
An evaluation-driven AI customer operations agent that resolves support tickets autonomously using tool calling and hybrid retrieval — with policy guardrails and human-in-the-loop escalation built in.
The client's support queue had a familiar shape: 60% of tickets were the same eight questions, answered by whoever was free, in whatever tone they happened to be in that day. They'd tried a chatbot before and killed it within a month because it hallucinated a refund policy that didn't exist and a customer took a screenshot. They wanted automation but didn't trust it — and honestly, they were right not to.
So the guardrails came first, not last. Every action the agent can take — issue a refund, look up an order, update an address — is a defined tool call against real systems, not the model improvising from training data. Hybrid RAG grounds every response in the current policy docs and order data, re-fetched per conversation rather than baked into a prompt that goes stale. A policy layer sits between the model's intent and the action itself, checking refund amounts against limits and order age against return windows before anything executes. Anything outside those bounds, or anything the model flags as uncertain, routes to a human with the full conversation history attached, not a cold handoff. We also built an evaluation harness that runs 200-plus scripted scenarios against every new prompt version before it ships, because "it seemed fine in testing" isn't a standard anyone should ship agents on.
Six weeks in, the agent was closing three out of five routine tickets without a human touching them, and — more importantly to the client than the automation rate — it hadn't caused a single guardrail incident. The eval suite has already caught two regressions before they reached production, which is the whole point of having one.