VoiceNest

VoiceNest – Empathetic Voice Companion Powered by AWS Lambda.

Comment

Inspiration

VoiceNest was inspired by the rising challenge of emotional loneliness in elderly individuals. Many live alone and find modern digital tools overwhelming. We wanted to create something simple, natural, and comforting — where the user only needs their voice to feel connected, heard, and emotionally supported.

What it does

VoiceNest enables elderly users to:

  • 🗣️ Speak naturally by recording their voice
  • 🧠 Get a thoughtful, AI-generated response with emotional awareness
  • 🌐 Receive replies in their own language
  • 🎧 Listen back to both their voice and the AI’s reply

It’s a truly voice-first interface — no typing, no complexity — with support for over 40 languages and sentiment detection.

How we built it

  • Frontend: Built with Next.js 14, shadcn/ui, Framer Motion, and MediaRecorder API for recording and playback.
  • Backend: Triggered via API Gateway into AWS Lambda, which:
  1. Transcribes audio (Amazon Transcribe)
  2. Detects language and sentiment (Amazon Comprehend)
  3. Translates input/output as needed (Amazon Translate)
  4. Generates empathetic replies (Cohere command-r-plus)
  5. Synthesizes voice (Amazon Polly)
    • Fully serverless infrastructure provisioned via Terraform, with CI/CD using CodePipeline and CodeBuild.

Challenges we ran into

  • Recording audio in proper format from frontend
  • Ensuring natural-sounding Polly voices across different languages
  • Managing fallbacks when Polly didn’t support a language
  • Syncing user voice and AI reply playback smoothly
  • Handling short, emotionally vague inputs in a meaningful way

Accomplishments that we're proud of

  • 🎉 Built and deployed a fully serverless voice AI app
  • 💡 Runs entirely within the AWS Free Tier
  • 🌍 Supports multilingual conversation for global accessibility
  • 🧓 Focused on elderly-friendly design with minimal interaction steps
  • 🚀 All code and infrastructure written during the Hackathon window

What we learned

  • Building a voice-to-voice AI experience is more complex than text workflows
  • Properly orchestrating AWS services in Lambda takes precision
  • Cohere’s LLM can match tone and emotion effectively when guided
  • UI feedback and loading states are key for trust in voice-based UX

What's next for VoiceNest

  • 🎭 Add emotion-based voice tones using Polly SSML
  • 📈 Store conversation history and sentiment over time
  • 📱 Launch a mobile-first PWA for real-world deployment
  • 📴 Support offline fallback using IndexedDB
  • 🤝 Explore partnerships with senior care organizations and retirement homes

Built With

Share this project:

Updates

Submission history