Hawking.ai

Stephen Hawking
Gel electrode for detecting EMG signal
OpenBCI Ganglion board

Inspiration

We found our North Star in Stephen Hawking. Despite possessing one of the greatest minds in human history, early hand-switch systems gave him ~15 words per minute (wpm), but as ALS progressed, he was reduced to 1–2 wpm using cheek sensors.

We built Hawking.ai to honour Hawking's legacy and solve the communication crisis he struggled with for decades.

What It Does

Hawking.ai is an autonomous communication platform that transforms raw biological signals into natural, context-aware conversations at conversational speed.

The User Experience
Users navigate a dynamic 4×4 semantic grid using just three discrete biosignals:

Move Right: single signal
Move Down: double signal
Select: long hold

Live Conversational Context
The system performs real-time transcription of the conversation partner using ElevenLabs' speech-to-text. This context feeds into our AI engine to predict the most likely response, dramatically reducing the number of inputs required to form complex sentences.

Voice Restoration
Users upload a voice sample as short as 10 seconds (from before their condition worsened). ElevenLabs' voice cloning technology synthesizes their biological voice, ensuring they speak to loved ones as themselves, not as a robot.

Conversational Flow
During testing, we achieved 25 words per minute. Compared to the 1–2 wpm limit Hawking faced in his final years, this represents an order-of-magnitude leap, moving users from passive observers to active participants in their own lives.

How We Built It

We engineered a full-stack, hardware–software loop optimized for minimum latency and maximum robustness.

Voice & Audio Pipeline – ElevenLabs as the audio foundation:

Speech-to-Text: real-time transcription of the conversation partner, enabling context-aware response prediction
Voice Cloning: instant synthesis of the user's biological voice from a 10-second sample
Audio Quality: clinical-grade clarity ensures emotional nuance is preserved

Signal Processing

WoodWide AI: filtering, low-pass filtering, subtracting reference signals, denoising
Reasoning API used

AI Orchestration & Prediction Engine

Architecture: FastAPI/Uvicorn WebSocket server coordinating the full pipeline
Intelligence Core: Google Gemini 2.0 Flash via OpenRouter
Chosen specifically for sub-250 ms time-to-first-token (TTFT)

State Management
Zustand handles complex grid navigation state with zero lag.

Hardware Layer

Ganglion Board with electrode sensors
Lab Streaming Layer (LSL) protocol for reliable, real-time signal ingestion

Frontend

Vite + React + TypeScript for performance
Framer Motion for smooth, non-distracting animations (the UI should disappear; only the thought should remain)
Responsive grid that adapts to different screen sizes and input speeds
Zero friction: every millisecond of latency is visible to the user as hesitation; performance was a core focus

Infrastructure
Docker containerization ensures reproducibility across hardware configurations and simplifies deployment to clinical settings without dependency conflicts.

Challenges Overcome

Our initial vision was a fully offline system: complete privacy, zero internet dependency.

We spent significant engineering hours optimizing local LLMs using ExLlamaV2, experimenting with 4-bit quantized versions of Qwen and Phi-3 running on consumer-grade GPUs. We successfully fit models into VRAM, but hit a critical wall: time-to-first-token (TTFT). Suggestions must appear instantly for our purposes. Local inference introduced a noticeable drag that broke the user's flow, transforming the experience from seamless thought completion to “waiting for a computer to talk for you.”

Strategic Pivot
We offloaded heavy cognitive lifting to the cloud, leveraging Google Gemini Flash 2.0 via OpenRouter. End-to-end latency dropped below 250 ms. The experience transformed from frustrating delay to genuine thought amplification.

Trade-off Mitigation
We designed privacy controls (local voice storage, optional on-device caching) to address privacy concerns while maintaining clinical-grade responsiveness.

Biosignal Noise
EMG is fundamentally stochastic: random muscle twitches, electrode drift, and electrical interference. Distinguishing intentional signals from noise is non-trivial.

Solution via WoodWide AI

Applied reasoning-based filtering to interpret context
Implemented adaptive thresholding that learns individual baseline noise profiles
Achieved <5% false positive rate

What We Learned

We learned that paralysis is not a loss of intelligence. It is a bandwidth problem. A high-capacity mind is trapped behind a low-capacity motor channel. Even four noisy binary signals can be enough, if amplified correctly.

WoodWide AI turns biological noise into usable meaning. It acts as a semantic decompression engine for the brain. Accessibility is not about simplifying interfaces; it is about maximizing information throughput.

Latency is a medical variable.
500 ms feels like using a machine.
200 ms feels like using your own body.

Assistive technology has the wrong value proposition.
The market asks: “Can you say ‘supercalifragilisticexpialidocious’?”
Users ask: “Can I sound like myself?”

What's Next for Hawking.ai

Phase 1: Clinical Validation (Q2 2026)
Partner with ALS research clinics for IRB-approved user testing. Gather longitudinal efficacy data against standard-of-care devices. This data is essential for regulatory submission.

Phase 2: Multimodal Sensor Fusion (Q3 2026)
Expand the biosignal stack to include:

Electrooculography (EOG) for eye-blink and eyebrow-raise detection
Additional EMG placements for redundancy as disease progresses
Adaptive switching logic that automatically routes to available residual motor control as the patient’s condition evolves

Phase 3: Privacy-First On-Device Learning (Q4 2026)
Develop an on-device learning loop that fine-tunes prediction models based on the user's vocabulary, slang, and historical communication patterns. Create a “digital twin” of their linguistic style that lives locally on the device, ensuring privacy while maximizing personalization.

Phase 4: Regulatory Pathway to Reimbursement (2027)
Map FDA Class II medical device classification pathway. Secure coverage under existing Medicare/Medicaid HCPCS codes for Speech Generating Devices (SGDs).

Goal
Transition from consumer gadget to prescribed, reimbursable medical necessity. This is how scale and equity are achieved, ensuring cost is not the barrier to access.

Bigger Picture
2.5 million people are waiting for communication technology that treats them as the cognitive equals they are. Hawking.ai is not a nice-to-have accessibility feature; it is a restoration of fundamental human dignity.

Stephen Hawking proved that a mind could still contribute to civilization even when locked in. Hawking.ai ensures that voice is never locked out again.

Every person deserves to speak. Every person deserves to be heard. Let's make it possible.

Check out the project here: https://github.com/ryanzhou147/NexHacks