ClearWord – AI-Powered Conversational Recovery Agent

• 

  Home Screen

• 

  Listening Screen

Inspiration

The idea for ClearWord came during a conversation at breakfast.

My grandaunt, who uses hearing aids, mentioned something that stayed with me:

"The hearing aids make everything louder, but I still can't understand what people are saying."

That observation changed how we thought about the problem.

Many assistive technologies focus on amplification. They help users hear more sound, but they do not necessarily help users recover meaning when speech is partially missed, unclear, fragmented, masked by background noise, or difficult to understand.

We realized the real challenge is often not hearing speech.

The challenge is recovering understanding.

ClearWord was created to explore whether AI could help users reconstruct what was most likely said after missing part of a conversation.

What it does

ClearWord is an AI-powered Conversational Recovery Agent designed for hearing-impaired users, hearing aid users, older adults experiencing age-related hearing loss, and people with auditory processing difficulties.

When a user misses part of a conversation, they can tap Replay Clearly.

ClearWord retrieves recent conversational context from MongoDB Atlas, combines it with the captured transcript, and uses Gemini to generate a clearer interpretation of what was most likely said.

The user is shown both the original transcript and the clarified version, providing transparency while helping them quickly recover meaning and continue the conversation naturally.

How we built it

ClearWord combines several technologies into a conversational recovery workflow:

• Gemini for contextual reasoning and clarification generation
• MongoDB Atlas for conversational memory and context retrieval
• MCP for standardized memory access
• Speech Recognition APIs for transcript generation
• Text-to-Speech for clarified audio playback
• React, TypeScript, and Lovable for the frontend experience
• Google Cloud Agent Builder for agent design and experimentation

Challenges we ran into

One of the biggest challenges was balancing accuracy with faithfulness.

When speech is unclear, AI models can sometimes over-correct or introduce wording that was never actually spoken. We spent significant time refining prompts, context retrieval, memory handling, and recovery logic to keep clarifications as close as possible to the original conversation.

We also faced challenges around latency, speech recognition quality, mobile usability, and accessibility-focused interface design.

Accomplishments that we're proud of

We successfully built a working proof of concept that combines speech recognition, conversational memory, contextual reasoning, and audio replay into a single accessibility-focused workflow.

The project demonstrates how conversational memory can improve understanding rather than simply generating transcripts.

What we learned

Through building ClearWord, we learned how memory systems, retrieval, contextual reasoning, and speech technologies can work together to support accessibility.

We also gained a deeper appreciation for how hearing loss affects everyday communication and how AI can potentially assist users in recovering understanding rather than simply amplifying sound.

What's next for ClearWord – AI-Powered Conversational Recovery Agent

ClearWord is currently an MVP and proof of concept.

Our next goal is to improve clarification accuracy, reduce latency, and expand the application into a dedicated mobile accessibility product.

We are particularly interested in exploring Bluetooth integration with hearing aids and assistive listening devices, enabling users to access conversational recovery directly from the tools they already use every day.

Our long-term vision is to build AI-powered accessibility technology that helps hearing-impaired individuals and people with auditory processing disorders participate more confidently in conversations.

Built With

• gemini
• google-cloud-agent-builder
• lovable
• mcp
• mongodb-atlas
• react
• speech-recognition-api
• text-to-speech
• typescript