Eleven

Sample replay. Claude drafts up to four replies and the earpiece reads them one at a time, so the person can answer without a screen.
Sample replay. Each suggested reply shows where it came from: the person's saved notes in Moss, or web and Maps results from Apify.
Sample replay. A care team request is filled in a browser, read back in the earpiece, and sent only after the person confirms.

Inspiration

In the United States alone, an estimated 35,000 adults are living with ALS in 2026, and more than 5,000 Americans are diagnosed every year. ALS progressively damages the nerve cells responsible for voluntary movement, potentially taking away a person's ability to speak or move while leaving their thoughts, awareness, and desire to connect intact.

For someone whose body can no longer respond to their intentions, even a simple conversation can become a painstaking process. Existing assistive technologies, such as eye-gaze systems and switch-based devices, can help, but communication can be slow and depends on physical abilities that the disease may eventually affect. Meanwhile, the world is rapidly moving toward AI agents that can search the internet, make plans, and carry out tasks. Yet these tools generally assume that the person controlling them can type, tap, or speak.

In high school, Khadim spent two years volunteering at a nursing clinic. Residents with ALS and cerebral palsy had sharp minds, but communicating their thoughts often required navigating an eye-gaze board or selecting one letter at a time. Their thoughts were complex, but the tools available to express them were limiting.

The most promising emerging alternatives include implanted speech brain-computer interfaces (BCIs), which have demonstrated remarkable potential but require brain surgery and remain under clinical investigation. We wanted to explore a different path: what if people could control AI agents using noninvasive EEG technology instead?

Inspired by Eleven from Stranger Things, a character who uses her mind to move and control things, we created Eleven to explore the intersection of brain-computer interfaces and AI agents. Our goal is to push the boundaries of what people with ALS and other motor disabilities may eventually be able to accomplish independently.

We treated this hackathon as a safety net for experimentation: an opportunity to explore what EEG can do, build a working system, and investigate whether people could use their brain signals to guide AI agents through everyday tasks.

What if losing the ability to move didn't mean losing the ability to act?

What it does

Eleven combines an EEG headband, an earpiece, and AI agents to help people communicate and initiate tasks without relying on a screen, speech, or invasive brain surgery.

Ask: A friend or family member asks a question out loud.
Understand: Claude acts as our orchestration layer, processing the conversation's context and coordinating retrieval from the person's memory, powered by Moss and SpacetimeDB. It generates up to four short responses in different tones, using saved notes and relevant external information when needed.
Listen: Cartesia converts the responses into speech and plays them through the earpiece, allowing the user to listen to each option. Its voice capabilities also support calls to consenting contacts.
Choose: The person selects a response using EEG-based motor imagery. The chosen response is then spoken aloud to the room.
Act: When a conversation leads to a task, the user can initiate an action through our Apify Actor or request a call to a consenting caregiver. Before anything is sent or submitted, Eleven reads the details back and requires a separate confirmation.

Four commands, without physical hand movement

The system uses four commands:

YES: Imagining a right-fist squeeze selects an option.
NO: Imagining a left-fist squeeze skips an option.
CONFIRM: A single jaw clench after a read-back authorizes the pending action.
STOP: A double jaw clench rejects the options, ends a call, or pauses the system.

Important: The fist squeezes are imagined motor movements, not actual hand movements. Jaw clenching is a physical muscle signal, not a brain signal.

Our decoder analyzes EEG signals associated with imagined movements. These commands are mapped to the user's intended actions, allowing the system to move through possible responses. During our first live headset session, Shashank successfully skipped an option with a left-fist squeeze and selected one with a right-fist squeeze.

Eleven remains experimental, and we have not yet tested it with people living with ALS.

Why motor imagery, not flashing screens?

Many brain-spelling systems use steady-state visually evoked potentials (SSVEP), which require users to focus on lights flickering at different frequencies. These systems depend on visual attention and can be unsuitable for people who have difficulty controlling their eyes or tolerating flicker.

Eleven explores motor imagery instead. Imagining a hand movement can produce changes in the brain's mu rhythm, typically around 8–12 Hz, over the sensorimotor cortex. Imagining a right-hand movement is associated with activity changes over the left motor cortex, near electrode C3; imagining a left-hand movement is associated with changes over the right motor cortex, near C4.

This approach lets users make selections without looking at flashing stimuli, keeping their eyes free to focus on the people around them. It does not eliminate the need for a device, and EEG motor imagery is not yet reliable for everyone, but it offers a promising alternative worth exploring.

Our decoder uses eight electrodes positioned over the motor areas and was trained on Shashank's own 337 trials. We evaluated filter-bank Common Spatial Patterns (CSP) and a Riemannian model using cross-validation to select an appropriate approach.

The brain signal selects the response; a deliberate jaw clench confirms an action. We distinguish these signals explicitly rather than presenting muscle activity as brain activity.

How we built it

EEG decoder — Python, BrainFlow, MNE, and scikit-learn: Converts imagined hand movements into YES and NO commands and detects jaw-muscle bursts for CONFIRM and STOP. The decoder runs locally and communicates through a WebSocket. Raw EEG data never leaves the laptop.
Brain visualization — Next.js, React, and React Three Fiber: Displays a 3D template brain, highlighting the C3 and C4 regions as the mu rhythm changes. A separate jaw-signal lane is labeled "not brain" to distinguish muscle activity from EEG.
SpacetimeDB — Real-time backend and memory coordination: Serves as our live backend, coordinating shared state across devices through subscriptions and reducers. This allows agents and interfaces to stay synchronized with current context. We also use server-controlled timing, heartbeat leases, and role-based checks to coordinate actions and pause the system if a station loses its connection.
Claude — claude-haiku-5-5: Acts as the orchestration layer, using structured outputs to coordinate agents and manage information retrieval and ingestion across our integrated tools. Without Claude, the individual components would be far more fragmented. It helps connect them into a coherent pipeline that can interpret context and coordinate the next step.
Cartesia — Speech and voice agents: Ink-2 handles incoming partner speech, while Sonic generates spoken responses for the earpiece and the room. Short-lived, 60-second tokens minted by the database help prevent browsers from holding long-lived credentials. Cartesia's voice-agent capabilities also support calls to consenting contacts.
Moss — Memory and retrieval: Provides access to the person's saved information and a per-question index of web results. This helps agents retrieve relevant context quickly, which is particularly important in a real-time conversational setting.
Apify — Web research, maps, and task execution: Provides our RAG Web Browser and Google Maps integrations for retrieving relevant live information. We also built a reusable Actor, Confirm-Gated Form Errand (stephensookra/confirm-gated-form-errand, Actor ID peGekcBzIp8LuhkPi). It fills a form, reads the actual values back from the page, and submits only after receiving a confirmation bound to a digest of the information the person heard. Other Apify tools support retrieval and actions throughout the workflow.
Tiger Data — Telemetry: Records content-free telemetry for commands and selections, including time-to-pick measurements, without storing the content of the conversation.
Automated testing: We built 1,897 automated tests across nine packages, running in CI on every push.

Where it could go wrong, and what we built for it

The decoder misreads a hand movement. A mistaken selection can choose the wrong words, but it cannot independently authorize an external action. Anything that leaves the room requires a read-back of the actual page and a separate confirmation.
The web is wrong or hostile. Claude works from retrieved information rather than treating arbitrary webpage content as instructions. The system is designed to avoid inventing places, times, prices, phone numbers, or people.
Motor imagery does not work for everyone. Each user needs an individually calibrated decoder. Motor commands are accepted only after a brain-signal test, and a jaw-only mode provides an alternative when motor imagery is not usable.

Challenges we ran into

The jaw, twice. During our first live session, swallowing produced stray single jaw clenches that triggered false STOP commands. We revised the mapping so a single clench confirms an action only immediately after a read-back, while a double clench stops the process.
Muscle activity can look like brain activity. Jaw clenching produces strong signals in the 20–30 Hz range that can overlap with beta-band EEG activity. We pause brain-signal interpretation while muscle activity is detected and visualize the jaw signal separately.
The wait. A web search can take approximately 17 seconds, which interrupts the natural flow of conversation. We designed the system to skip external searches for small talk and simple yes-or-no questions when retrieval is unnecessary.

Accomplishments that we're proud of

The complete loop ran live: A question moved through search, memory retrieval, Claude orchestration, earpiece playback, response selection, and an errand that was read back, confirmed, and submitted.
The confirmation gate held: A CONFIRM command issued during the read-back was ignored. A confirmation issued after the completion chime was accepted, and the system reported "Sent" 3.9 seconds later.
A real call worked: Eleven called a teammate, identified itself as an AI, and communicated the request.
Reliable timing: The scan clock ran a median of 2 ms late across 76 steps. The dead-man pause activated 1,501–1,502 ms after the last heartbeat in all four tests.
The decoder worked offline: On held-out trials, the decoder classified left-hand imagery correctly in 21 of 29 trials, right-hand imagery in 19 of 23 trials, and rest in 10 of 15 trials. These results are preliminary and based on Shashank's individual data; they do not establish clinical accuracy or generalizability.
A reusable Apify Actor: We published a confirm-gated form workflow that previews an action, lets a person confirm the information they heard, and submits only after authorization.

What we learned

The hardest part of a brain-computer interface is not just decoding the signal. It is building everything around it: timing, confirmation, synchronization, personalization, and safe behavior when something fails.

We learned that EEG signals vary between individuals, making calibration essential. A model that works for one person may not work for another, and a decoder that performs well in a controlled test may struggle in everyday conditions.

We also learned that EEG remains an experimental and technically challenging field. That limitation became an opportunity: we had to investigate the signals ourselves, compare modeling approaches, and build a custom pipeline rather than rely on a ready-made solution.

Most importantly, assistive technology cannot be evaluated by whether a demo works once. It must be evaluated by whether a person can trust it, correct it, stop it, and use it safely in the real world.

What's next for Eleven

We want to take Eleven beyond a hackathon prototype and toward a system that people with ALS and other severe motor disabilities can meaningfully evaluate and shape.

Our next steps are to improve individualized EEG calibration, test reliability across more participants, reduce selection latency, and work with people with ALS, caregivers, and clinical researchers to understand what actually makes communication and daily tasks easier. We also want to validate our confirmation safeguards under realistic conditions before expanding the range of actions the system can perform.

We recognize that motor imagery will not work for everyone and that noninvasive EEG has real limitations. Our goal is not to claim that we have solved mind-controlled computing, but to explore what becomes possible when AI agents are designed around people who cannot use conventional interfaces.

As students, we want to push the boundaries responsibly. Eleven is a step toward a future where physical limitations do not automatically mean losing access to communication, digital tools, or greater independence—and where the people building AI agents consider not just what their technology can do, but who gets to control it.