CallPilot — AI Operations Agent for Real Phone Workflows

Inspiration

Every business runs on phone calls — booking appointments, chasing quotes, qualifying leads, coordinating services. Chatbots stop at text; the real work lives behind a phone number. When CALL-E opened the "Your Code Is Calling" hackathon, the idea clicked instantly: don't build another AI that talks. Build an operations layer that is given a job and completes it over the phone.

What it does

CallPilot turns a structured business task into an autonomous phone workflow:

Task → AI Plan → CALL-E Call → Adaptive Conversation → Structured Result → Action

A user creates a mission ("find the earliest appointment"), picks one of 6 agent templates (Appointment, Quote, Lead Qualification, Availability, Service Coordination, Follow-Up), sets a safety mode, and reviews the generated execution plan. After explicit approval, CallPilot places a real call through CALL-E, tracks it live on an operations dashboard (status timeline, event feed, progress toward the goal), and converts the conversation into structured data — availability slots, prices, requirements, Q&A coverage, confidence score, AI summary, and a recommended next step. Not a transcript dump. Actionable data.

Safety is a first-class feature: Information-Only, Approval-Required, and Autonomous (low-risk only) modes, with guardrails that forbid unauthorized purchases, contracts, payments, and sensitive-data disclosure. The agent always identifies as AI. Demo/sandbox runs are watermarked MOCKED everywhere — nothing simulated is ever presented as a real call.

How I built it

• Frontend: Next.js 14 + TypeScript + Tailwind — landing page, mission wizard, live call dashboard with polling event feed, results page with conversation intelligence, history, analytics, and a Developer Mode that exposes the exact CALL-E request, agent config, events, and output (secrets redacted).
• CALL-E integration (the core): an isolated service layer (services/calle/calleClient.ts) imports the official @call-e/calle SDK at runtime — CalleClient.calls.create() to start, calls.get() to poll honest status snapshots — with a raw Developer API fallback (POST /v1/calls, GET /v1/calls/{id}) and a terminal webhook receiver (/api/calle/webhook) with at-least-once deduplication on the CALL-E-Event-Id header. The live feed emits only observed status transitions — conversation facts are never invented.
• Backend: Next.js API routes + Zod validation, per-mission agent prompt generation, schema-first result extraction.
• Storage: PostgreSQL (Neon) mirroring a full relational schema (tasks, calls, call_events, agent_runs, call_results) — required because serverless instances share no disk; local dev uses a file store.
• Deploy: Vercel (production) + GitHub. Secrets live only in env vars.

Challenges I ran into

1. Honesty in the event feed — my first version simulated progress events on a timer, even for live calls. I rewrote it to non-blocking start + real status polling so the dashboard only shows what CALL-E actually reported.
2. Serverless amnesia — missions created on one Vercel instance 404'd on the next. Fixed with a durable Postgres store and row-locked transactions.
3. Pooled-connection gotchas — prepared statements break under PgBouncer transaction pooling (prepare: false fixed it), and a half-migrated JSON mapping briefly stored stringified arrays (fixed with self-healing decode + stack-trace-driven debugging).
4. Vercel SSO wall — deployment protection gated the public demo API; relaxed to preview-only via the Vercel API.

Accomplishments I'm proud of

• A complete, judged-tested loop: 9/9 checks passing on the public URL (pages, plan, live run, persistence, analytics, webhook).
• A product that feels like a startup, not a demo — missions, safety gates, developer transparency.
• Zero fake claims: every mocked surface is labeled, proven in the video's Developer Mode segment.

What I learned

Goal-driven calling beats scripted IVR trees — you describe the outcome and let CALL-E adapt. The hard part isn't the call, it's result discipline: schema-first extraction is what turns talk into completed work. And serverless + stateful workflows demand a real database from day one.

What's next

Multi-recipient batch missions, mid-call approval callbacks, Supabase Auth + team workspaces, call recording links, and more agent templates (collections follow-up, vendor onboarding).

Built With

• call-e
• neon
• nextjs
• node.js
• postgresql
• rest-api
• tailwind-css
• typescript
• vercel
• webhooks
• zod