Inspiration

A person with aphasia can know exactly what they want to say and still be unable to retrieve the word. That moment can stop a conversation, shift control to the other speaker, and gradually erode confidence. Aphasia affects roughly 2–4 million people in the United States, with an estimated 100,000–180,000 new cases each year. [1][2]

Speech-language pathologists already use cueing strategies — such as a beginning sound, a written letter, or a semantic prompt — to help a person retrieve a word rather than simply supplying it. Evidence also suggests that retrieval practice can produce more durable naming gains than repetition alone, although outcomes vary with the individual and the underlying impairment. [3][4][5]

The gap is timing. Therapy happens in a clinic, while word-finding breakdowns happen at the kitchen table, in a store, on the phone, or in the middle of a family story. Nudge began with one question: What if the smallest useful clinical cue could appear exactly when the word gets stuck — without interrupting eye contact or speaking for the person? Prior work with people with aphasia found head-worn vocabulary prompts feasible and suggested that glanceable support can help users stay engaged with a conversation partner. [6]

What it does

Nudge is an AI-powered word-finding aid that runs on everyday display glasses. It listens to the flow of a conversation, recognizes when the wearer asks for help or appears to stall, predicts likely missing words from recent context and a personal word list, and displays the least revealing cue first.

The interaction is intentionally simple:

Listen: The glasses' microphones stream the wearer's speech for real-time transcription.
Ask: The wearer taps the temple for help; an experimental assist mode can also detect a stalled sentence.
Predict: A language model ranks likely words using the recent transcript and personally relevant vocabulary such as names, medications, and places.
Cue: The display shows the first letter and blanks. Another tap reveals part of the word, a third reveals the whole word, and a swipe moves to another prediction.
Focus: In a group conversation the microphones hear everyone. An experimental wearer-voice gate uses loudness — the wearer's voice is consistently louder at the temple than a partner or bystander — so only the wearer's speech can trigger a hint, while other voices still feed the context the prediction reads.

Nudge includes Practice Mode for structured naming exercises and Conversation Assist for real-life speech. In Practice Mode, each word walks a graded ladder — the sentence frame, its first letter, the letter with a picture, then the shown answer — so help always arrives before the word is given away, and the session summary counts how many words came back without help. The display stays blank when help is not needed. The goal is not to generate a sentence or replace the wearer's voice; it is to give just enough support for the wearer to finish the thought independently.

A typical moment might be: "I need to call my… um…" Nudge first displays M _ _ _. If that is enough, the wearer says "Maya" and continues. If not, the wearer can reveal more letters or swipe to the next likely word.

How we built it

We built Nudge as an end-to-end smart-glasses application on the Even Realities G2 platform. The G2 provides binocular micro-LED displays, a four-microphone array, temple touch controls, and an open plugin workflow through Even Hub; the application logic runs through the paired phone while the glasses handle display, audio input, and gestures. [7][8]

The prototype pipeline is:

Even G2 microphones → streaming speech-to-text → recent conversational context + personal word list → fast language-model ranking → progressive letter cue on the lenses.

The hackathon build uses ElevenLabs Scribe for speech-to-text, a server-side prediction layer using Gemini 3.5 Flash-Lite, and a device-agnostic display interface that renders the cue hierarchy on the G2. API keys remain on the server rather than the glasses, and the current design does not store audio. A companion phone app hosts the personal word bank, pause and sensitivity controls for auto-hints, and a session log the wearer can export to review with a clinician. The team also built gesture controls (tap, swipe, double-tap, and an experimental head-tilt: a sideways tilt acts like a tap on the cue ladder and looking up or down moves through the candidate list, tuned on real IMU data for users who cannot tap), immediate segment completion on tap, alternative-word swiping, and experimental automatic stall detection.

We treated latency as a core product requirement rather than a backend metric. The team benchmarked candidate models on the same stalled-sentence set, measured prediction and end-to-end timing, and selected the model that consistently fit the conversational time budget. Modern streaming speech-to-text systems are designed for partial, low-latency transcripts, making this type of live interaction technically practical. [9][10]

Challenges we ran into

Conversation-speed latency: A useful answer that arrives after the conversation has moved on is not useful. We shortened the path by finalizing the speech segment as soon as the wearer taps, warming the server path, and benchmarking models under the same prompt and test cases.
Predicting an unspoken word: Multiple completions may fit the same sentence. We return a ranked top three, blend recent context with the user's personal vocabulary, and let the wearer swipe rather than forcing a single guess.
Detecting a real stall: Pauses, filler words, and unfinished phrases can indicate word-finding difficulty, but they also occur in ordinary speech. Early automatic hints produced both true detections and false alarms, so auto-hints remain experimental and can be paused by the wearer.
Separating the wearer from a partner: The microphones hear the whole conversation. Early device tests showed a useful loudness gap between the wearer's voice and a partner across the table, so each utterance is tagged wearer or other by its measured level. A partner's words still enrich the prediction context, but they can never trigger — and can even cancel a pending — hint. The gate ships behind a flag while we tune it across rooms.
Preserving agency: The technically easiest product would simply display or speak a full answer. That would undermine the central use case. The harder design problem was to provide graded help while keeping the user in control of the sentence.
Accessibility beyond aphasia: Stroke can also cause hemiplegia, visual-field loss, motor planning difficulty, and working-memory limitations. Smart-glasses co-design research warns that gesture-heavy interaction and inaccessible displays can exclude the people the device is meant to serve. [11][12]
Privacy and social trust: A conversational wearable must feel safe to the wearer and everyone nearby. Nudge chose camera-free hardware, minimizes display activity, keeps credentials off-device, and is designed not to retain audio.

Accomplishments that we're proud of

In 24 hours, the team moved from a clinical insight to a working prototype on real Even G2 glasses — not only a phone mock-up. Live speech can trigger a context-aware word prediction, and the selected cue appears in the wearer's field of view with tap and swipe control.

The current engineering evaluation reported in the deck achieved the intended word in the top three for 31 of 31 team-written stalled sentences, compared with 68% for a word-list-only baseline. Median model prediction time was 0.76 seconds, the warm live-server path was approximately 0.95–1.22 seconds, and the simulator measured about 1.5 seconds from tap to the first cue. These are small internal engineering benchmarks, not evidence of clinical effectiveness, but they demonstrate that the core interaction can operate at conversational speed.

The team is also proud that the product uses AI to support — not replace — human communication. The progressive cue ladder, blank-by-default display, personal vocabulary, no-camera platform, and user-controlled fallback all reflect the same principle: the wearer should remain the author and speaker of the thought.

What we learned

Speed is part of the therapeutic experience: A slower, more elaborate model performed well on answers but missed the interaction window. For Nudge, the best model is the one that is accurate enough and fast enough to preserve the conversation.
Less assistance can be more useful: Written and part-word cues are meaningful ingredients in naming treatment, and retrieval practice can strengthen later access to words. Starting with one letter respects both the evidence and the wearer's independence. [13][14]
Personal context matters: A generic model can complete a sentence, but family names, medications, favorite places, and recurring routines are often the words that matter most. The personal word list is both a performance feature and a path toward genuinely individualized support.
Automation must remain interruptible: Automatic hints can reduce effort, but false positives can be distracting or patronizing. The wearer needs an obvious way to pause, dismiss, or request help.
Aphasia-accessible design must be co-designed: Research consistently points toward simple interfaces, customization, accessible instructions, and direct involvement of people with aphasia throughout development. Engineering feasibility is only the first checkpoint. [15][16]
The cue ladder can become an outcome signal: Logging whether a person needed one letter, half the word, or the full word could create a lightweight longitudinal measure of real-world retrieval support between therapy visits.

What's next for Nudge

The next phase is to move from a strong engineering prototype to a product shaped and tested by the people who will use it.

Co-design and usability: Test at home with approximately 10 people with aphasia and five speech-language pathologists. Evaluate gesture accessibility, visual presentation, setup burden, comfort, false hints, and the ability to recover from wrong predictions.
Reliability and privacy: Improve noisy-room performance, speaker identification, offline or degraded-network fallback, consent controls, data minimization, and transparent handling of transcripts.
Personalization tools: Build a simple companion experience through which users, families, and clinicians can add important names, medications, places, and daily-life vocabulary without creating a complex interface.
Clinical pilot: Conduct the proposed eight-week, multiple-baseline pilot with about 20 adults with chronic post-stroke aphasia. Measure practiced and unpracticed naming, word retrieval in conversation, communication confidence, quality of life, and change in the level of cue required.
Clinical workflow: Give speech-language pathologists a concise view of practice and cue-use trends, while ensuring technology adds to — not replaces — ongoing clinical care. [17]
Regulatory pathway: Launch with carefully limited communication-support claims, confirm the appropriate FDA pathway with regulatory counsel, and pursue therapeutic claims only after usability and clinical evidence support them.
Broader access: After validating the post-stroke aphasia use case, explore primary progressive aphasia, traumatic brain injury, and dementia-related word-finding difficulty, with separate research for each population rather than assuming one design fits all.

Nudge does not speak over the user. It gives the smallest possible hint, at the exact moment it is needed, so the user can keep the conversation — and the voice — their own.

References

Aphasia — ASHA Practice Portal, American Speech-Language-Hearing Association. https://www.asha.org/practice-portal/clinical-topics/aphasia/
Quick Statistics About Voice, Speech, and Language — NIDCD, NIH. https://www.nidcd.nih.gov/health/statistics/quick-statistics-voice-speech-and-language
Errorless, Errorful, and Retrieval Practice for Naming Treatment in Aphasia [FILL: full citation + link]
Errorless, Errorful, and Retrieval Practice for Naming Treatment in Aphasia (second source) [FILL: full citation + link]
Aphasia — cueing strategies [FILL: full citation + link]
Designing Conversation Cues on a Head-Worn Display to Support People with Aphasia [FILL: full citation + link]
Even Hub Documentation — Glossary (Even G2 hardware). [FILL: link]
Even Hub Documentation — Overview (color depth). [FILL: link]
ElevenLabs — Realtime Transcription (STT) API, Scribe v2 Realtime. https://elevenlabs.io/docs/api-reference/speech-to-text
ElevenLabs — Create a Token (Realtime STT). [FILL: link]
Making smartglasses accessible: perspectives and prototypes [FILL: full citation + link]
Framework Analysis of the iReadMore App Co-Design Process [FILL: full citation + link]
Identifying the components of a successful spoken naming treatment [FILL: full citation + link]
Towards a Theory of Learning for Naming Rehabilitation: Retrieval Practice [FILL: full citation + link]
Co-design of a web accessibility browser extension for people with aphasia [FILL: full citation + link]
A scoping review — PMC, PubMed Central, NIH [FILL: full citation + link]
Priorities and design principles for a mobile health app for people with aphasia [FILL: full citation + link]