CallFlow

Inspiration

Most conversational voice agents operate as brittle, open-loop scripts: dial a number, recite a prompt, and fail silently when encountering unexpected friction like busy schedules, ambiguous receptionist replies, or gatekeeper pushback. When typical bots hit these roadblocks, they either abort completely or blindly execute the exact same flawed prompt on the next call. We built CallFlow to bridge this gap, elevating voice agents from rigid auto-dialers into an autonomous, closed-loop telephony platform that diagnoses conversational breakdowns and systematically self-improves with each call.

What it does

CallFlow autonomously executes telephone calls via the CALL-E Model Context Protocol (MCP), evaluates the outcome, and retries with refined strategies until the objective is verified:

Closed-Loop Strategy Mutation (v1 → v2): When a call misses its objective, CallFlow analyzes the transcript, refines its approach (e.g., switching from open-ended questions to confirmation-first schedule framing), and re-engages automatically.

11-Class Failure Taxonomy: Categorizes dialogue friction into distinct failure modes—such as PROVIDER_UNAVAILABLE, MISUNDERSTANDING, GATEKEEPER_BLOCKED, and COMPLIANCE_STOP—and triggers targeted recovery playbooks.

5-Dimension Scoring Rubric: Scores calls across Task Fulfillment, Completeness, Correctness, Quality, and Efficiency to ensure outcomes meet quality standards (≥ 0.70 threshold) before marking tasks complete.

Three-Tier Persistent Memory: Stores run-level state in Task Memory, maintains vendor quotes and reliability in Provider Memory, and records prompt heuristic success rates across runs in Strategy Memory.

Dual-Mode Telephony: Seamlessly toggles between live carrier dialing (CALLE_MODE=real) and a deterministic, multi-turn simulator (CALLE_MODE=simulated) with real-time SSE waveform visualization.

Built-In Telephony Ethics: Enforces mandatory identity disclosure, E.164 phone whitelisting, zero financial commitments without human approval, and immediate opt-out blacklisting upon receiving a Do-Not-Call request.

How we built it

Full-Stack Architecture: Built on Next.js 15 (App Router), TypeScript 5.7, Tailwind CSS, and Prisma 6 ORM backed by SQLite/PostgreSQL.

CALL-E MCP Client: Implemented native integration with CALL-E’s MCP tool suite (plan_call, run_call, get_call_run) alongside an automated discovery mechanism for local CLI authentication tokens.

State Machine & Telemetry: Engineered a modular TaskPlanner and CallOrchestrator finite state machine (FSM) paired with typed Server-Sent Events (SSE) to stream live transcripts, speech activities, and waveform telemetry directly to the UI.

Automated Test Suite: Formatted full verification coverage through Jest/Vitest unit tests for the diagnostic evaluator and end-to-end multi-call adaptive mutation loops.

Challenges we ran into

Protocol Synchronization: Streaming multi-turn speech events from CALL-E polling loops into responsive, low-latency UI waveforms without race conditions or dropped transcript packets required tuning SSE buffer intervals.

Deterministic Failure Diagnosis: Distinguishing between genuine contractor unavailability (PROVIDER_UNAVAILABLE) and ambiguous date misunderstandings (MISUNDERSTANDING) demanded rigid JSON schema validation and conservative heuristic classifiers over messy spoken dialogue.

Balancing Persistence Tiers: Decoupling short-term conversational context from persistent strategy heuristics so that lessons learned in one calling session benefit future tasks without carrying over task-specific sensitive data.

Accomplishments that we're proud of

Achieving a fully functional, self-healing telephony loop where an initial failed booking attempt automatically mutates into an upfront, structured framing that secures a verified booking on the subsequent attempt.

Implementing a zero-cost deterministic simulation mode that mimics multi-turn conversational quirks, allowing full offline testing without burning carrier minutes.

Delivering a clean, enterprise-grade mission control dashboard with zero build errors and 6/6 passing automated unit and integration tests.

What we learned

Phone conversations with front-desk staff are highly non-linear; open-ended prompts frequently fail, whereas front-loading specific timeframes and constraints dramatically increases first-call resolution rates.

Implementing strict compliance guardrails (such as instantaneous COMPLIANCE_STOP handling) directly into the orchestration state machine is critical when running autonomous voice bots on live networks.

What's next for CallFlow

Multi-Provider Parallel Dispatch: Expanding the orchestrator to conduct concurrent multi-vendor inquiries and negotiate optimal terms across competing quotes simultaneously.

Inbound Voice Webhooks: Supporting autonomous inbound handling for callbacks requested during initial outbound outreach.

Dynamic Few-Shot Injection: Automatically injecting winning prompt phrasing from Tier 3 Strategy Memory into CALL-E call plans dynamically based on target industry classification.