Inspiration

We grew up in immigrant families, so we've seen how much of a burden urgent English notices are to deal with, and how often fixing one means hours on hold and confusing phone menus. That’s why we built Avocado, a multi-agent application integrated in WhatsApp that serves as a caring assistant, deep researcher and document verifier, and a phone operator that makes long calls so you don’t have to.

What it does

Avocado is a multi-agent WhatsApp assistant: it explains your letter in your language, checks that it's real, and makes the call for you.

Summary and Verification: you send a photo of the letter and get an explanation in your native language within seconds. The letter is cross-checked against official sites for scam signs.
Phone Tree Navigation: it dials only the verified official number and only with your recorded consent, works through the phone menu, waits on hold, and never answers identity questions for you.
Handoff and Live Translation: it rings you when a person answers, and the dashboard shows a live transcript translated into your language with suggested replies.

How we built it

Three agents (orchestrator, research, phone) coordinate through Postgres jobs and events, and every decision streams live to the dashboard.

Research Agent: Claude Opus with web search runs a fast pass that explains the letter and a deep pass that verifies it and finds the official number.
Phone Agent: Twilio and Pipecat run the call, Cartesia handles speech-to-text and text-to-speech, Claude does the reasoning, and a state machine tracks menu → hold → person answers → bridge.
Translation Agent: everything is in English internally and translated only at the edges using a self-updating translation book of sourced legal terms.

Challenges we ran into

Real phone lines and low-resource languages are messy.

Hold vs. human: we match repeated hold recordings without calling a model and send Claude only the audio it hasn't heard before.
Explicit Support: We built in translation books and leveraged Opus to support many languages, in addition to dedicated handling for unique language attributes (e.g. right-to-left text for Dari/Farsi).
Bridge timing: we ring the user quickly enough that the representative doesn't hang up, but briefly enough that voicemail can't answer.

Accomplishments that we're proud of

The WhatsApp, phone, and research agent flow works end to end: photo → explanation → verification → consent → phone menu → hold → user connected.

The safety rules (only verified numbers, consent before calling, never impersonating the user) are enforced in code, not prompts.

What we learned

Agents are more trustworthy when you can see what they're doing easily.

Logging every decision as an event gave us a debugger, a demo, and transparency for users from the same data.
Claude handles the reasoning, and code makes the safety-critical decisions.