Inspiration
Imagine being in a conversation, knowing exactly what you want to say, but not being able to say it out loud.
By the time you've typed your response, the conversation may have already moved on.
For people who cannot reliably produce audible speech, communication isn't just about finding a way to generate words. It's also about timing, spontaneity, personality, and feeling included in everyday conversations.
That got us thinking: What if we could make communication feel less like operating software and more like simply joining a conversation?
Existing text-to-speech and AAC tools already do important work. We wanted to explore something complementary: an experience that combines silent articulation, intelligent assistance, personalized speech, and quick conversational interactions, all within reach while someone is using their Mac.
That's how TACIT was born.
Your words. Your voice. Beyond speech.
What it does
TACIT is a macOS communication companion that helps people who cannot speak aloud express themselves during everyday conversations and video calls.
Instead of requiring someone to type everything, TACIT offers another way to communicate: silently mouth a sentence into your webcam, review what the AI understood, and choose when to speak it aloud.
The experience lives inside a native, interactive MacBook notch interface designed to stay accessible without interrupting whatever you're doing.
But communication is about much more than converting words into audio.
TACIT includes:
- Silent-to-Voice: Visual speech recognition interprets lip movements without requiring audible speech. Users can review and correct the result before speaking.
- My Voice: Choose from available macOS voices, preview them privately, and customize supported speech settings. Your voice should be your decision.
- Hold My Turn: Instantly communicate something like "One moment, I'm responding," without losing what you were composing.
- Express Yourself: Save personalized phrases for reactions, questions, humor, and everyday conversation.
- Type Anytime: A familiar typed communication option remains available whenever silent recognition isn't suitable.
- Conversation-first notch: A compact, fluid interface that keeps communication controls close without taking over the screen.
Once someone approves a message, TACIT synthesizes it and routes the audio into a virtual microphone that can be selected in ordinary calling applications, including Google Meet.
TACIT never automatically speaks an AI interpretation. The user stays in control of what gets said.
How we built it
TACIT combines visual speech recognition, native macOS development, real-time communication, and speech synthesis.
For silent recognition, we integrated a pretrained Auto-AVSR visual speech recognition model running through PyTorch with Apple's Metal Performance Shaders (MPS). A browser-based camera workspace uses MediaPipe to process facial landmarks and prepare visual input for recognition.
The recognized words are then sent to TACIT's backend, where they become reviewable candidates rather than automatically spoken messages.
On the desktop side, we built a native SwiftUI and AppKit companion that lives around the MacBook notch. We focused heavily on state-aware animations, fast interactions, keyboard accessibility, and preserving what someone is trying to communicate.
For speech output, TACIT uses macOS speech synthesis through a custom Swift voice helper. Users can select supported system voices, while BlackHole 16ch provides a virtual microphone route for calling applications.
We also built explicit speech authorization, utterance tracking, interruption handling, and emergency Stop controls. These are especially important when a recognition mistake could cause someone to say something they never intended.
The result is a system connecting webcam input, AI recognition, user review, personalized speech, and ordinary communication applications.
Challenges we ran into
Silent lip-reading is much harder than we expected.
Many sounds look nearly identical on someone's lips, and pretrained visual speech recognition models don't necessarily perform well when someone mouths words without making sound.
Our own testing exposed substantial recognition errors. That forced us to rethink an important product decision: should TACIT automatically say what the model predicts?
We decided no.
Instead, we built a review-and-correction workflow where the user has the final say. Making that experience fast and comfortable became just as important as improving the model itself.
Another major challenge was getting synthesized speech into ordinary video calls. We had to coordinate voice synthesis, virtual audio routing, the native interface, and the recognition service while keeping latency low.
We also encountered tricky reliability problems involving outdated recognition results, duplicate speech requests, and interrupted communication. These reinforced how important it is to preserve someone’s intended words and avoid unexpected speech.
Finally, designing the notch was its own challenge. We wanted something beautiful and fluid, but every animation and interaction needed to serve a real communication purpose.
Accomplishments that we're proud of
We're particularly proud that TACIT evolved beyond a technical lip-reading demonstration into a more complete communication experience.
We built a native macOS companion that brings together multiple ways of expressing yourself, rather than forcing everyone into one input method.
Our speech pipeline also became substantially more responsive. In local tests, reviewed speech reached the virtual microphone in approximately 152 milliseconds after pressing Speak, compared with roughly 0.9–1.2 seconds in our earlier implementation.
We implemented selectable voices, private previews, fast expressive phrases, and a way to hold your place in a conversation without sacrificing an unfinished thought.
Just as importantly, we built safeguards around a fundamental principle:
An AI model can suggest the words, but it should never decide what someone says.
What we learned
The biggest lesson was that communication is not the same thing as speech generation.
A text-to-speech engine can read a sentence aloud. But participating in a conversation involves much more: finding the right moment, responding naturally, expressing emotion, and having confidence that the words being spoken are actually yours.
We also learned that accessibility features need to be designed around trust. A fast response is valuable, but not if it says the wrong thing. A beautiful interface matters, but not if someone struggles to use it.
And we learned that there isn't one universal way to communicate. Silent articulation won't work for everyone, which is why we believe alternative input methods and user choice need to remain central to TACIT.
What's next for TACIT
We see TACIT as the beginning of a larger idea: an intelligent, personal communication layer that helps people participate more naturally across their digital lives.
Our next steps include:
- Personalized visual speech recognition: Adapt recognition to individual users' articulation patterns rather than relying entirely on a general pretrained model.
- Context-aware communication assistance: Explore opt-in meeting context that can help someone identify conversational openings or draft a response, without speaking on their behalf.
- More personal voice options: Investigate consent-based custom voices and voice preservation through technically and legally supported methods.
- Broader accessibility: Explore additional input methods for people who cannot silently articulate or comfortably use a camera.
- Real user evaluation: Work with AAC users and accessibility communities to understand practical needs, test usability, and improve the product based on lived experience.
Built with Cursor and Grok
Cursor and Grok played a major role in building TACIT. We used them extensively for the heavy lifting throughout development, from implementing core functionality and the native macOS experience to debugging, testing, and refining the final product. Both helped us rapidly turn our idea into a working application during HackUMass XIV.
We haven't validated TACIT with AAC users yet, and reliable silent recognition remains an open challenge. Those are important parts of the work ahead.
Ultimately, we don't want TACIT to replace someone's personality with an AI-generated one.
We want to give people more ways to express who they already are.
TACIT — Your words. Your voice. Beyond speech.
Built With
- appkit
- apple-metal
- auto-avsr
- avfoundation
- blackhole
- css
- html
- javascript
- macos
- mediapipe
- node.js
- python
- pytorch
- swift
- swiftui
- typescript
- websockets
