Inspiration
I hate speaking on the phone and in meetings, but sometimes I still have questions. It is often hard to find answers when you are not confident to speak up in a meeting.
What it does
UNMUTE is a real-time meeting voice proxy that uses ElevenLabs Scribe v2 for live transcription and Flash v2.5 for instant text-to-speech generation. It allows users to type thoughts into a web UI that automatically fire into the meeting when specific "trigger words" are detected in the conversation.
How we built it
Audio Capture & Routing: Used BlackHole 2ch on macOS to create a virtual audio loopback, allowing the app to "hear" meeting audio from apps like Zoom or Google Meet while simultaneously routing the generated proxy voice back into the meeting.
Real-time Transcription (STT): The backend maintains a persistent WebSocket connection to ElevenLabs Scribe v2, streaming raw PCM audio chunks from the BlackHole device for near-instant speech-to-text.
Intelligence & Triggers: Implemented a logic layer that scans the incoming live transcript for user-defined trigger words; when a match is found, it initiates a high-fidelity ElevenLabs Flash v2.5 text-to-speech generation.
Seamless Playback: Generated audio is saved locally and triggered via a subprocess call to macOS's native afplay utility, ensuring the voice is delivered to the meeting with minimal latency.
Frontend Synchronization: The UI was built with HTML5/CSS3 and vanilla JavaScript, using Socket.IO to provide users with a live transcript, a visual "auto-fire" countdown, and a dynamic response queue.
Challenges we ran into
This was my first time working with ElevenLabs, so my first challenges relate to setting that up, but once that was accomplished ElevenLabs ran smoothly. The main challenge I had was relating to having the AI "speak", this led me to use BlackHole 2ch, which allows the audio to be properly played, while also being able to listen to the conversation at the same time.
Accomplishments that we're proud of
This is the first hackathon that I have done so honestly I am proud that this is working in any state, let alone how effective it appears to be.
What we learned
I was insanely impressed with how easily it was to set up a voice clone of myself, requiring under 30 seconds of my voice to be able to do an impressively similar voice. Also was my first 24 hour, this was a new experience making a project in such a short time frame.
What's next for UNMUTE
- Remove the need to switch from virtual mic to real mic when wanting to speak using own voice
- Switch static keywords to a context based implementation for triggering audio response
- Lastly, switching from a macOS dependancies, to more cross platform system
Log in or sign up for Devpost to join the conversation.