Caller AI — We wait. You talk.

Caller AI safely navigates a reviewed support phone tree and hold queue, then calls the verified customer back only when a live representative is ready.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Caller AI — We wait. You talk.

Inspiration

Waiting on hold is a tax on people's time. The useful moment is the human conversation at the end—not navigating menus or listening to hold music. Caller AI gives that time back while keeping the customer in control.

What it does

A customer chooses a reviewed company route and explicitly authorizes a call. Caller AI navigates the bounded IVR flow and waits in the queue. When it detects a live representative, it stops navigating, calls the customer's verified number, and bridges the two humans. It does not impersonate the customer, handle credentials, authorize purchases, negotiate, or continue speaking after the customer joins.

How we built it

Each approved route is represented as a Call Graph: observed prompts plus allowed transitions for one company and one department. GPT-5.6 receives only the current IVR observation and proposes one typed action under a strict JSON Schema. Deterministic policy then validates that proposal against the graph, user authorization, callback ownership, business hours, quotas, budget, and sensitive-data gates before Twilio can execute anything. If a condition is missing, the system fails closed.

Twilio Voice handles calls, DTMF, callbacks, and the final human-to-human bridge. The service and UI are TypeScript. OpenAI API storage is disabled. Codex was our implementation partner for the Build Week expansion: continuous IVR audio processing, faster transfer detection, durable transfer evidence, hardened callback acceptance, a second reviewed route, tests, documentation, and the live public evaluator.

Try it safely

The public /demo is intentionally interactive without exposing arbitrary outbound dialing. Judges can enter any fictional English IVR prompt and see a live GPT-5.6 structured proposal, the independent policy verdict, and the exact bounded Twilio command that would be eligible for execution. Multi-step handoff, OTP-stop, and no-answer fixtures provide reproducible Call Graph replays. The evaluator places zero phone calls.

Real dialing requires an authenticated user, verified callback number, explicit call-bound consent, reviewed destination, quotas, signed Twilio webhooks, and an audit receipt. Removing those gates from a public contest link could enable harassment, unconsented callbacks, premium-rate misuse, and unexpected telecom charges. Sensitive prompts such as OTP, PIN, password, card, and SSN requests stop locally before any model or provider request.

Challenges we ran into

The main challenge was separating model judgment from execution authority. We also had to handle ambiguous IVR audio, race conditions during representative pickup, webhook retries, no-answer callbacks, and restoration after a page closes—without letting the agent expand its own scope.

Accomplishments that we're proud of

  • A private-pilot end-to-end UPS handoff with a verified callback and durable receipt
  • An independently enforced typed-action boundary
  • A live judge-facing GPT-5.6 evaluator with zero telecom side effects
  • Explicit refusal paths for credentials and unsupported requests
  • Meaningful post–July 13 Build Week extensions documented in the repository history

What we learned

Voice-agent quality is not just latency or model accuracy. The decisive product feature is a narrow, inspectable authority boundary: AI may interpret and propose; deterministic software decides and executes.

What's next

We will expand reviewed Call Graph coverage, add consented pilot users and businesses, improve multilingual prompt handling, and use Twilio conversation intelligence to measure transfer quality while preserving minimal data retention.

Built With

Share this project:

Updates