PhoneScout , AI Phone Research Agent

Inspiration We've all been there: you need to know if a restaurant can fit 15 people on Friday, whether a pharmacy has flu shots today, or what a mechanic charges for a smog check. You search online — the website hasn't been updated since 2019. The Google listing says "open" but you call and it's closed. The menu doesn't mention group reservations.

The phone is still the most reliable way to get real-time information from small businesses. But making 5-10 phone calls, waiting on hold, asking the same questions over and over — that's 30-60 minutes of your life you'll never get back.

We built PhoneScout to automate exactly this. Instead of you spending an afternoon on hold, an AI agent makes the calls, has natural conversations, asks your specific questions, and returns a compiled report with verified answers.

What It Does PhoneScout runs a three-step research pipeline:

Search — Finds relevant businesses using the Google Places API, filtering for those with phone numbers Call — Uses CALL-E's SDK to dial each business. The AI agent holds a natural, polite conversation, asks your specific questions, adapts to the person's responses, and extracts structured data Compile — Aggregates results across all calls, ranks them by relevance and completeness, and returns a structured report It comes with 4 built-in use cases with tailored prompts and extraction schemas:

🍽️ Restaurant — Group size accommodation, date/time availability, dietary options, deposit requirements 💊 Pharmacy — Medication/service availability, pricing, insurance acceptance, hold-for-pickup 🔧 Mechanic — Service quotes, earliest appointments, service duration, loaner car availability 💪 Gym — Day pass availability, pricing, hours, free trials, facilities Plus a custom mode for any question to any business type.

How We Built It Architecture:

User → CLI / Python API → PhoneScout Orchestrator ├── BusinessSearch (Google Places API) ├── CallHandler (CALL-E Python SDK) ├── PromptBuilder (use-case templates) └── ResultsCompiler (ranking & formatting)

text

Key technical decisions:

CALL-E Python SDK for all phone calls — we use calls.create() with custom agent prompts and extract_schema for structured data extraction, then wait_for_completion() to get results including transcript summaries and extracted JSON data
Google Places API for business discovery — text search filtered by type, then details API to get phone numbers. Businesses without phone numbers are skipped
Use-case-specific prompts — each use case has a carefully crafted prompt that guides the AI agent through the conversation naturally: be concise, respect their time, offer to call back if they're busy, thank them at the end
Structured extraction schemas — JSON schemas per use case so CALL-E returns machine-readable data alongside the conversational summary (e.g., can_accommodate_party: true, estimated_cost: "$45")
Result ranking — businesses are scored by rating, structured data completeness, and whether the answer was affirmative, so the most relevant results appear first
Sequential calling with pauses — calls are made one at a time with a 2-second pause between calls to be respectful to businesses

Integration with CALL-E:

The core integration lives in call_handler.py:

call = self.client.calls.create(
    to=business.phone_number,
    agent_prompt=prompt,
    extract_schema=extract_schema,
    timeout=self.config.timeout_seconds,
    metadata={
        "business_name": business.name,
        "source": "phone-scout",
    },
)
result = call.wait_for_completion(timeout=self.config.timeout_seconds)
CALL-E is called at runtime for every research query — it's not just referenced or imported, it's the engine that makes the whole project work. Every phone call goes through CALL-E's SDK, and the structured extraction is what makes the results machine-usable rather than just conversational.

Challenges We Ran Into
Business search quality — Google Places sometimes returns businesses without phone numbers, or with outdated numbers. We handle this by filtering out businesses without phone numbers at the search stage, and gracefully handling call failures in the results compilation.
Prompt engineering for phone conversations — A prompt that works for text chat doesn't work for phone calls. The agent needs to be more concise, more polite, and handle real-world situations like voicemail, busy signals, and people who don't want to answer questions. We iterated on the prompts to balance getting the information with being respectful of people's time.
Structured extraction reliability — Not every call yields clean structured data. Some businesses give partial answers, or the conversation goes in unexpected directions. We handle this by making the extraction schema optional and always returning the conversational summary as a fallback, then scoring results based on data completeness.
Rate limiting and politeness — We can't blast 10 businesses with calls simultaneously. We implemented sequential calling with pauses between calls, and a configurable concurrency limit for cases where parallel calls are appropriate.
CALL-E SDK nuances — Learning a new SDK under hackathon time pressure is always a challenge. The CALL-E documentation and example skills were helpful for getting the basic call flow working, and we filed feedback on areas where we hit friction.
What We Learned
The phone is an untapped data source. Everyone focuses on web scraping, but the richest information about small businesses lives behind a phone call. CALL-E makes this data accessible programmatically for the first time.
Conversational AI needs different design than chat AI. On a phone call, there's no scrolling back up. The agent needs to be concise, ask one question at a time, and adapt if the person goes off-script. This is a different prompting skill than chat-based agents.
Structured extraction is the key to making phone calls useful for developers. A transcript is nice, but {"available": true, "price": "$25", "hours": "9am-9pm"} is actionable. CALL-E's extract_schema feature is what bridges the gap between conversation and code.
CALL-E is genuinely different from other AI phone platforms. The SDK-first approach, the structured extraction, and the MCP/Skill support make it feel like a developer tool, not just a phone automation service. This is the right abstraction layer for building phone-based agents.
What's Next
More use cases — Hotel availability, event venue research, contractor quotes, school enrollment inquiries
Calendar integration — When a restaurant confirms availability, automatically create a calendar hold
Multi-step workflows — "Find a restaurant, book it, then call a babysitter for that night"
SMS fallback — If a call goes to voicemail, send a text with the question
Web dashboard — A UI for creating research queries and viewing historical reports