A Cross Functional Architecture for Agentic Calls

GIF
Media

A huge amount of real-world information is still locked behind phone calls. Businesses, government offices, customer-service teams, and local organizations often require people to navigate IVRs, wait on hold, and ask questions manually.

We wanted to explore a simple idea: What if a phone call could become an executable information interface? That led us to build A Cross-Functional Architecture for Agentic Calls — an architecture where a natural-language objective can become a real phone conversation and ultimately structured intelligence.

What it does

CallBot Detective transforms a natural-language mission into an agentic phone investigation.

The system can:

Understand a user's objective and break it into targeted questions. Identify and prepare the call target. Initiate a real outbound phone call using CALL-E. Navigate phone systems and IVRs. Conduct a conversational interaction with a real person. Capture the conversation as evidence. Extract answers and organize them into structured information. Produce an investigation-style report with the questions, answers, confidence, transcript, and call information.

Instead of simply generating an answer with AI, the system goes out into the real world and gathers information through a phone call.

How we built it

We designed the project as a cross-functional architecture rather than a single AI component:

Natural Language → Mission Planning → Target Resolution → Agentic Call Execution → IVR Navigation → Human Conversation → Evidence Capture → Structured Intelligence → Verified Report

The backend uses Python/FastAPI with SQLite for investigation persistence. CALL-E provides the real phone-call infrastructure, while an AI planning layer converts natural-language missions into actionable questions.

The frontend was built with Next.js, React, TypeScript, and Tailwind CSS, providing the investigation workflow, live call states, history, and professional report interface.

The architecture deliberately separates planning, telephony, conversation, evidence, and reporting so that each function can evolve independently.

Challenges we ran into

The biggest challenge was connecting an AI-driven workflow to the unpredictability of real telephone systems.

We had to handle:

Real outbound calls and call-state transitions. IVR and phone-system navigation. Long-running calls and asynchronous results. Temporary API timeouts while retrieving completed calls. Persisting call results reliably into SQLite. Keeping the frontend synchronized with the backend during a live investigation. Distinguishing a temporary API failure from an actual failed call.

We also discovered that a successful phone conversation does not automatically mean the final structured result is immediately available. Designing around that asynchronous behavior became an important part of the architecture.

Accomplishments that we're proud of

We're proud that this went beyond a simulated AI conversation.

We successfully built an end-to-end architecture capable of taking a mission from natural language to an actual phone conversation and structured investigation workflow.

We are especially proud of:

Making a real outbound CALL-E call. Building an agentic workflow around the call rather than simply wrapping a chatbot. Designing clear functional boundaries between different components. Creating live investigation states from dialing through analysis. Building a structured investigation report and history system. Handling failures and transient API problems instead of treating every error as a failed investigation. Creating a foundation that can be extended to many different real-world information-gathering tasks. What we learned

We learned that agentic systems become much more interesting when they interact with the physical world.

A phone call introduces problems that don't exist in a normal web-based AI workflow: waiting, interruptions, IVRs, uncertainty, human conversation, incomplete information, and asynchronous results.

We also learned that good agentic architecture is not just about making an AI model smarter. It is about creating reliable boundaries between intent, planning, execution, evidence, and verification.

Most importantly, we learned that a phone conversation can be treated as an information interface — one that AI agents can potentially operate just like they operate APIs and websites.

What's next for A Cross-Functional Architecture for Agentic Calls

Our next goal is to move from a prototype into a more capable general-purpose phone intelligence platform.

We want to explore:

More robust IVR and call navigation. Better conversation planning and adaptive questioning. Stronger evidence and answer verification. Multiple calls for cross-checking information. Automatic target discovery and phone-number resolution. Richer investigation reports. Support for customer operations, research, service discovery, accessibility, and business intelligence. A reusable architecture where different specialized agents can work together across the phone-call lifecycle.

The long-term vision is simple:

If the web can be programmed through APIs, we believe the phone network can become another programmable interface for intelligent agentsInspiration