Inspiration
Ayo did not start as a safeguarding project. It started with Sabi, a conversational voice agent startup we built that teaches children mathematics over an ordinary phone call. After building Sabi, we received feedback from child counselors that there was a deeper need: children need a companion they can open up to. Many children have no safe adult to talk to, and existing helplines can be intimidating to call directly.
That feedback shaped Ayo. We set out to build more than a therapist-like voice: an intelligent harness behind the conversation that connects to existing child abuse support lines through our dashboard, so that the information a child shares can be passed on safely to the people trained to act on it.
What it does
How we built it
Ayo is built on the full-duplex telephony stack, rebuilt for trauma-informed youth support:
- Dual-protocol telephony: Twilio Media Streams over WebSocket for standard phone numbers, and Asterisk AudioSocket over TCP for low-latency GSM networks, so children can reach Ayo from basic phones.
- Full-duplex, real-time voice: Gemini Live powers the conversation with instant barge-in handling. We wrote an in-memory G.711 mu-law codec and a 24 kHz to 8 kHz resampler with no external C dependencies.
- Clinically grounded prompting: The agent's behavior is rooted in Psychological First Aid, the NICHD Child Forensic Interview Protocol, and Bruce Perry's Neurosequential Model (Regulate, Relate, Reason), with open invitations, de-blaming language, and no false secrecy.
- An intelligent harness behind the voice: A background coordinator triages topic and risk after each turn, retrieves reviewed clinical guidance (RAG over pgvector with Cohere embeddings), and publishes structured guidance that the voice agent pulls without slowing the call.
- Crisis escalation to real humans: Defense-in-depth guardrails short-circuit on abuse disclosures, self-harm, and immediate danger, routing to verified lines (Nigeria's 112, the Cece Yara Foundation, 988 internationally), with consent-based emergency SMS to responders. Through our dashboard, this connects to existing child abuse support lines so trained counselors receive what matters.
- Privacy by design: Voice-based caller verification, encrypted consent-gated memory limited to a fixed vocabulary (it never stores free-form history of disclosures), deep encryption of voice data, audit logs, and retention limits.
Challenges we ran into
Making the voice emotive and empathetic. The hardest problem was getting the agent to pick up cues from a child's conversation and respond with genuine warmth. We switched from Gemini to ElevenLabs to solve this, and built a comprehensive prompt dataset to recognize abuse patterns from conversational cues and steer the conversation, all while making the child feel safe.
Escalating to the right people. Sending an SMS to the relevant authorities only after the agent harness understood that a situation was critical required careful decision logic: escalating too eagerly breaks trust, and escalating too late defeats the purpose.
Credits and latency. We ran out of Gemini tokens faster than we could blink. We kept latency low by optimizing how frequently our code called the model, while still keeping the voice agent empathetic.
Accomplishments that we're proud of
- Making the voice truly emotive. Getting a phone-line AI to sound like a caring companion rather than a machine is our biggest achievement.
- Building an end-to-end agent harness, from live voice conversation all the way to making the decision to escalate a critical situation to the relevant authorities.
- Voice authentication as an added layer of protection for children, so returning callers are recognized and their information stays theirs.
What we learned
- Bi-directional (full-duplex) voice calls. Streaming audio both ways over a real phone network, with interruptions handled gracefully, was new territory for us.
- Making a voice emotive. Empathy is not just the words the model chooses; it is the voice engine, the prompt design, and the timing, all working together.
- Listening to domain experts early. We thought we were building an education tool. Counselors showed us the real gap, and that feedback redirected the project.