VoiceFlow: AI Assistive Speaking

Inspiration

We kept noticing that text-to-speech tools sound nothing like the person typing. If you are losing your voice to something like VCD or your speech gets interrupted by a condition like Tourette's, getting handed a flat robot voice feels like losing even more of yourself. One of our teammates lives with VCD, so this wasn't theoretical for us, and we wanted something that actually sounded like you. Since current solutions are too complicated (they require multiple apps and messing with your microphone) we formulated a simple solution- based right from your browser with no account required. This allows you to communicate simply and efficiently without making other people just keep looking at the message boxes.

What it does

Record about a minute of yourself talking, and VoiceFlow clones your voice. After that, type and it speaks in your voice. If you have never been able to speak, you can grab one of premade voices instead. The innovative part that makes it so accessible and sets us apart is Live Rooms. Open a room, send the link, and whoever you're talking to just leaves the tab open in the background. So no chat to chat and they don't need to keep waiting for your messages. You type while in the Live-Room, hit Speak, and it plays on their end almost instantly unless you set a specific delay.

How we built it

We used FastAPI backend to integrate ElevenLabs for cloning and TTS, with the API key stored locally so it never touches the browser. Frontend is just HTML/CSS/JS, no framework, kept deliberately simple. Live Rooms run on WebSockets. Creating a room gives the host a private token and a 6-character code. The host's socket broadcasts text and audio to anyone connected on the listener side. Everything lives in memory and expires automatically.

Challenges we ran into

Researching the current market solutions and coming up with a hassle-free and super accessible one took a long time, then there were issues with browsers not letting websites autoplay audio, we resolved that by adding a button. There were also tons of connection issues between Live-Room listener and speakers as FastAPI's Query() dependency is not reliable for WebSocket routes, so the token never went through but pulling it straight from query_params fixed it in like two lines, after debugging for some time.

Accomplishments that we're proud of

Honestly just seeing the whole thing work end to end, seeing someone type a sentence in their own voice, and have it play out loud on someone else's browser in another tab with no app or sign-up, felt pretty great the first time it worked. Also proud we didn't skip the accessibility features. Our project actually can be used by people suffering from these issues as is, and a UI that doesn't feel like every other generic AI demo out there.

What we learned

We believe this project helped us get into the door of using NLP and play around with computational linguistics to actually accomplish something that would benefit the world socially. It was not just about learning ElevenLabs API, researching the whole process and learning how to code the project, it was also about learning what some people suffer from and understanding how they feel.

What's next for VoiceFlow: AI Assistive Speaking

Our plan is to try to work with ElevenLabs or other institutions to fund its development for three or four more months to actually launch this for professionals and general public who face these vocal diseases or cannot speak at all, our mission will be the same and this will continue to be a non-profit with easy sign-up and will maintain the same level of accessibility. We plan to add more features and improve the voice cloning and TTS experience in the future and launch this as an actual website for people to use when it has reached a certain level of quality.