CoCo | Devpost

Landing/Onboarding
Home Dashboard
Voice Recorder
Conversation Log
Practice with AI Partners
Track your Progress

Inspiration

Several of us have tutored and worked with children, and over time we noticed something: neurodivergent kids struggled to feel welcome in group settings in ways that felt social, not academic. They were smart, engaged learners, but navigating conversations and reading the room, was hard for them. We wanted to build something that could help.

Special education programs at K-12 schools are chronically underfunded, which means neurodivergent students often don't get the structured support they need to develop social skills. The tools that exist are either locked behind expensive therapy or designed for clinical settings, not everyday practice.

Stanford built an AI social coach focused on teaching empathy to autistic children, and 71% of users showed measurably increased empathy responses after using it. We took that a step further. That told us the approach works: a personal AI coach you could practice with anywhere, anytime, without a therapist in the room.

What it does

CoCo is an iOS app that gives you a personal AI coach for social skills. It works across three tracks.

Track 1: Daily Interaction Log

Record a voice memo of any real conversation. A local AI model (via ZETIC) processes it entirely on-device and surfaces a sentiment score, social cues, strengths, and specific improvement areas.

Track 2: AI Coach

Based on your logs, your coach breaks down patterns, explains what happened, and lets you practice that exact situation. You can also pick from general personas:

Alex: small talk, low-stakes catch-ups
Morgan: behavioral questions, professional pressure
Jordan: first introductions, talking about yourself
Sam: setting boundaries, disagreeing respectfully

Tap Reflect after any session for one strength, one improvement, and one exact line to try next time.

Track 3: Progress Dashboard

Tracks sentiment scores, improvement streaks, and skill growth over time. Earn points and badges for logging interactions, completing sessions, and hitting personal bests.

How we built it

Layer	Technology
iOS framework	Swift / SwiftUI
Android framework	React Native + Expo (see challenges)
Voice AI	ElevenLabs Conversational AI + LiveKit WebRTC
On-device analysis	ZETIC Melange (social cue detection, sentiment)
Android AI	Gemini (see challenges)
Backend / Auth / Storage	Supabase
Navigation	Native iOS Navigation / Expo Router (file-based)

The practice flow gives each persona a custom system prompt built from the user's onboarding profile. Their name, anxiety level, goals, and focus areas all feed directly into how the AI behaves in that session.

The recording flow pipes audio into ZETIC Melange running on the device's NPU, then pushes the structured analysis to Supabase. No audio is sent to the cloud.

Daily scenarios are generated by cross-referencing the user's profile against their prior sessions, so suggestions are always grounded in recent gaps rather than generic advice.

Challenges we ran into

This was our first real 36-hour in-person hackathon, and these were the walls we hit.

Finding the right idea: There are too many generic hackathon projects out there. We wanted to build something with a real impact, and the education sector kept coming up. It usually focuses on large groups, but the people who need the most help are individuals, specifically the ones working on skills that no classroom teaches.

ZETIC + Swift: We wanted on-device inference for privacy reasons but couldn't get ZETIC working with React Native. After reaching out to one of their advisors, we learned we'd need to switch to Swift. Privacy was non-negotiable for us given that users are logging personal conversations, so we built the main app in Swift for iOS. We'd like to bring ZETIC to the Android version in the future.

External API error handling: ElevenLabs and other APIs were returning generic error codes with no useful context, so we had to build out our own error handling layer to get reliable behavior.

Free tier limits: ElevenLabs' free tier kept hitting its cap mid-session, so we had to cycle through API keys to keep things running during the demo

Accomplishments that we're proud of

Built a full end-to-end voice practice loop where you go from tapping to live audio in under 5 seconds, through the full reflect and save flow
On-device social cue analysis that runs entirely on the user's phone with zero audio ever leaving the device
A persona system that adapts to the user's stated goals and recalibrates daily based on what recent sessions show
A UI that actually feels calm and human, because we knew that if the app felt clinical or cold, people with social anxiety just wouldn't open it
A Stripe-inspired character design system that keeps the visual language consistent and polished across every screen, making the app feel intentional rather than thrown together.

What we learned

Real-time voice AI on mobile is genuinely hard. Latency alone is a problem, but WebRTC state management and graceful error recovery on top of that means every edge case needs its own solution
On-device inference with ZETIC Melange is fast enough to feel instant, and that matters a lot when the pitch of your app is that nothing leaves the phone
The interface itself has to feel safe. For a mental health-adjacent app, design is not a layer on top of the product, it is part of what makes the product work at all

What's next for CoCo

Android support: The Android app is currently in React Native. We plan to port it to native Kotlin so we can bring the ZETIC bridge, and full on-device analysis, to Android users
Spaced repetition: Resurface specific scenarios on a schedule based on long-term retention curves, so weak areas actually get drilled over time
Therapist mode: A read-only dashboard where a coach or therapist can review a client's progress alongside them