Mobile Assist

An intelligent voice AI powered by Deepgram that listens beyond words, detects real time customer frustration, pauses gracefully, and auto-creates support tickets so no unhappy customer slips through.

Comment

Inspiration

Customer support AI often focuses on resolution, not emotion. We noticed that most voice bots continue conversations even when users are frustrated, leading to silent churn. The idea was to build a voice AI that doesn’t just respond, but knows when to step back and bring in human help.

What it does

Mobile Assist is an emotion-aware voice AI that listens beyond words. It detects frustration, dissatisfaction, and confusion in real time during conversations.

If negative sentiment is detected, it gently pauses the interaction and assures the user that a human executive will follow up. Every interaction, resolved or escalated, automatically generates a support ticket with emotional context for operational visibility.

How we built it

We built Mobile Assist using:

  • Voice interaction powered by Deepgram
  • Real-time sentiment monitoring via Deepgram Audio Intelligence
  • Function calling to trigger external actions
  • Automated ticket creation using Trello

The system continuously analyzes tone and conversational signals while the AI speaks, enabling parallel sentiment detection and real-time escalation.

Challenges we ran into

  • Running sentiment analysis in parallel without interrupting conversations
  • Defining thresholds for dissatisfaction detection
  • Designing a soft-stop mechanism instead of abrupt termination
  • Ensuring tickets captured both issue and emotional state
  • Handling complaint scenarios with faster escalation sensitivity

Accomplishments that we're proud of

  • Built real-time frustration detection into voice AI
  • Enabled graceful AI-to-human handoff
  • Ensured 100% interaction tracking via auto-ticketing
  • Created emotional context tagging for support teams
  • Prevented unresolved negative experiences from going unnoticed

What we learned

  • Resolution alone does not equal satisfaction
  • Emotional intelligence is critical for conversational AI
  • AI needs escalation boundaries
  • Operational visibility improves trust in automation

What's next for Mobile Assist

  • Smarter escalation prediction
  • CRM integrations
  • Multilingual sentiment detection
  • Analytics dashboard for emotional trends
  • Adaptive response behavior based on user mood

Built With

Share this project:

Updates