VoiceEver

Preserve the way you say it.

Inspiration

What would you want your daughter to hear on her wedding day, your son at graduation, or your spouse on an anniversary if you couldn't be there to say it?

For people living with ALS, speaking and typing can become increasingly difficult as the condition progresses. We wanted to address two challenges: helping people communicate in the present while preserving their voice, personality, and memories for the future.

Our fictional patient case follows Alex Chen, a biology teacher living with ALS. As speaking becomes more difficult, he wants to continue sharing classroom updates while preserving messages for his daughter Maya's future milestones.

What it does

Voice is an AI-powered voice preservation and assistive communication platform designed around people living with ALS.ver

  • Voice Preservation: Record original audio and video keepsakes, complete guided personality interviews, and create an authorized AI voice clone using selected recordings.
  • AI Message Generation: Use Gemini to draft personalized messages for birthdays, graduations, weddings, and other milestones using saved memories and familiar expressions. Users can review and approve messages before generating speech.
  • Glide Typing: An alternative text input method that lets users trace words across an on-screen keyboard instead of pressing individual keys. A custom gesture decoder ranks possible words.
  • 3D Voice Experience: Generate speech in the user's enrolled voice and pair it with an animated 3D likeness.
  • Family Keepsakes: Turn meaningful messages into family film previews combining generated audio and text, allowing loved ones to keep the moments the speaker chooses to preserve.

How we built it

Frontend (Next.js + React + TypeScript + Tailwind): Onboarding, voice recording, Studio, Glide Typing, and interactive controls.

Speech AI (ElevenLabs): Speech transcription, authorized voice cloning, and text-to-speech generation.

AI Personalization (Gemini): Drafts personalized messages using saved memories and expressions, with optional candidate reranking for Glide Typing.

Glide Typing (Custom TypeScript): Gesture recognition system that compares continuous keyboard strokes against possible word paths and ranks candidate words directly in the browser.

3D Avatar (Tripo + Three.js + React Three Fiber): Generates and displays an animated 3D likeness alongside speech.

Storage (IndexedDB + Supabase): Stores recordings and generated takes locally, with optional private cloud backup.

Family Films (Remotion): Creates video-style previews combining message text and audio.

Challenges we ran into

One of our biggest challenges was integrating multiple AI services into one cohesive experience. Connecting voice cloning, speech generation, timing information, and 3D avatar animation required coordinating several different systems.

We also wanted to distinguish original recordings from AI-generated speech, ensuring that users could preserve their authentic voice recordings rather than relying entirely on a generated replica.

Another challenge was developing Glide Typing from scratch. We had to build a gesture decoder that could interpret continuous keyboard strokes, rank possible words, and account for imperfect input.

Accomplishments that we're proud of

We're proud of bringing together voice preservation and assistive communication in one platform. VoiceEver supports both everyday communication, such as Alex sharing classroom updates, and meaningful future messages for his family.

We're also proud of developing Glide Typing as a custom input method rather than relying entirely on existing keyboard tools, and integrating it with our voice generation and 3D avatar experience.

Most importantly, we built VoiceEver around the idea that preserving someone's voice means preserving more than just the words they say.

What we learned

Building VoiceEver taught us how to integrate speech AI, language models, custom gesture recognition, browser-based recording, and 3D rendering into one application.

We learned that voice preservation involves more than voice cloning. Original recordings capture the pauses, laughter, and emotion that synthesized speech cannot fully reproduce.

We also learned how important accessibility, user consent, and control are when building technology intended to support people with changing communication needs.

What's next for VoiceEver

Improve Glide Typing through testing with people who use assistive communication. Refine avatar expressions and speech synchronization. Expand personal vocabulary support and develop secure family access for shared keepsakes.

VoiceEver is currently a prototype, and further testing with users and accessibility specialists would be needed before real-world use.

Built With

Share this project:

Updates

Submission history