DuoCode Customer Agent

Inspiration

Customer support AI often breaks down at the moments businesses care about most: refunds, payment questions, private customer data, unsupported promises, and requests that should go to a human. A chatbot can sound helpful, but a real business agent needs to know when to answer, when to use tools, when to cite evidence, and when to stop.

We built DuoCode Customer Agent to explore a safer version of support automation: an agent that can help customers quickly while leaving behind enough evidence for a merchant to trust, audit, and improve it.

What it does

DuoCode Customer Agent is an auditable customer-service agent for merchant support. Customers can ask questions about policies, orders, services, refunds, privacy, and support workflows. The agent answers from approved merchant knowledge, uses scoped business tools, records citations, and creates human-review tickets for risky situations.

For the Arize track, we added observability around the agent run. Redacted traces show retrieval, model response, evidence checks, tool usage, and handoff behavior. Gemini acts as a structured shadow reviewer: it reviews the audit bundle and labels whether a run is an auto-resolve candidate, needs a human, is unsafe, lacks evidence, or has a policy/tooling gap.

How we built it

The project is a TypeScript monorepo with a Next.js customer-agent UI, Convex as the product truth layer, and API routes for customer-agent chat, audit replay, trace export, and Gemini review.

Convex stores conversations, messages, tickets, citations, tool executions, run events, Gemini shadow reviews, and external export records. Arize receives redacted OpenInference-style traces through an OpenTelemetry export path. Gemini reviews redacted Convex audit bundles and returns schema-validated JSON verdicts.

The admin console shows the full support context: transcript, ticket state, evidence status, citations, tools, and Gemini analysis metadata.

Challenges we ran into

The hardest part was keeping the agent useful without making it reckless. It is easy to build an agent that answers confidently. It is much harder to build one that can prove what evidence it used, what tool it called, why it escalated, and what it refused to do.

We also had to keep a clean boundary between observability, review, and product truth. Arize helps explain what happened. Gemini helps review whether it looked safe. Convex remains the replayable source of truth for the business.

Accomplishments that we're proud of

We built a working support loop with source-grounded answers, tenant-scoped tools, evidence gates, human-review tickets, audit replay, Arize trace export, and Gemini shadow review.

We are especially proud of the safety boundary. Gemini can classify and explain runs, but it cannot approve refunds, close tickets, suppress handoff, mutate customer state, or override deterministic gates. The admin UI explicitly displays Gemini output as analysis metadata, not a final decision.

We also validated the system with runtime evals, red-team checks, manual UI tests, and human/Gemini review pairings.

What we learned

Real-world agents need more than a capable model. They need evidence, permissions, replay, observability, and clear human-control boundaries.

We learned that Gemini is very valuable as a reviewer for agent behavior. Instead of replacing human oversight immediately, it can help a business understand which cases may eventually be safe to automate and which must stay under human review.

What's next for DuoCode Customer Agent

Next, we want to turn the shadow-review layer into a production-ready assisted-review workflow. Low-risk, read-only, well-grounded cases can be ranked for faster review, while refund, payment, privacy, account, and contract-impacting requests remain human-controlled.

Longer term, DuoCode Customer Agent can become a safer support automation platform for small businesses: fast enough to help customers immediately, but auditable enough for merchants to trust.

Built With

• arize
• convex
• gemini-api
• google-cloud-agent-platform
• next.js
• openai-file-search
• openai-responses-api
• openinference
• opentelemetry
• promptfoo
• react
• stripe
• typescript
• zod