Inspiration

Exploring new ways to interface with computers is incredibly exciting. Innovative methods can allow users to escape the traditional constraints of the mouse and keyboard, and push to make life much more futuristic, accessible, and fun.

Companies like Neuralink have achieved the impossible when it comes to making computers accessible to everyone. The last frontier of this problem is making the solution open source and cost efficient.

What it does

Neuro North lets users accomplish text-based tasks, without moving their fingers, or opening their mouth. Anything where a user interacts through the use of natural language presents a valid use case for the Neuro North. We've integrated with AI agents so you can launch tasks and ask questions entirely with your brain.

It works by associating ranges of characters ([A-F], [G-L], [M-R], [S-Z]) to oscillating patterns on the user's computer screen using SSVEP (a non invasive BCI technique where specific hz frequencies are assigned to specific areas of the screen e.g. targets for letter ranges).

When the user focuses on one of these ranges, their brain activity appears with the same frequency of oscillation as the pattern, allowing us to record what range of letters they are targeting.

Using this mechanism, the user can spell out words on their screen letter-by-letter. By using context-aware decoding, we can predict the user's exact intended phrase.

This involves the use of a local model along with a prefix tree. When a user wants to spell the word "RARE" the model knows that its working with the ranges M-R, A-F, M-R, A-F.

As the model receives more data on what each letter could be it dwindles down the possible options of what the word could be by referencing the possible combinations present within the general english dictionary. By the end of the word the model reaches its top two most probabilisitic results which are given back as auto complete suggestions.

We also use the IMU on the non-invasive BCI to relate certain movement actions like head swing left being "cycle ranges", head swing right being "space / move onto the next word", head swing up and head swing down being autocomplete suggestions.

Composio

We used Composio as the action layer for our BCI agent, giving it secure delegated access to Google Calendar and Gmail so it could create real calendar events and Gmail drafts without building custom OAuth or app-specific integrations.

Browser Base

We achieved agentic browser control via Neuro North's control plane. This was done using Browser Base's API and we used GPT 4o to make decisions/reason about what API calls to make based on vague input like "click the second link", etc.

OpenAI

We used OpenAI as the reasoning layer for our agentic BCI, interpreting noisy brain-spelled text into structured actions like calendar creation, Gmail drafting, and browser control.

We also used local GPT-2 for real-time word prediction and sentence-level beam search inside the speller UI, helping turn uncertain letter selections into usable text.

Codex acted as our development teammate throughout the build, helping us plan, implement, merge, debug, test, and polish the OpenAI + Composio agent workflow into a working demo.

How we built it

BrainFlow
NeuroPawn Knight with IMU

Decoding part:

English lexicon, using prefix trie to search
using GPT-2 as a way to decode ranges based on the past context (GPT-2 is not post-trained, so it acts as a general autocomplete rather than a traditional AI assistant). The range decoder produces a constrained shortlist, then a locally hosted GPT-2 ranks only those valid candidates
beam search over the entire sentence. we keep the 8 strongest sentence hypotheses, allowing later words to repair earlier mistakes

Challenges we ran into

Non-invasive headsets have severe limitations for collecting data
Biggest: collecting a reliable signal
many words share the same ranges, so relying on the EEG range collection alone cannot produce the correct reconstructed/decoded words. We ran into lots of cases where different words have the same sequence of ranges (nice -> ride, tacos -> years, what -> that). Greedy decoding is not enough to recover the intended words. This led us to sentence-wide beam decoding that can rely on future context to decode current word
Accuracy versus latency and interaction cost The stronger language processing was meaningfully slower:
- Bigram: 51 ms/action
- GPT-2: 153 ms/action
- Greedy GPT-2: 31.76 seconds for the autonomous probe
- Sentence-beam GPT-2: 44.92 seconds, about 41% slower
Galvanic isolation

Accomplishments that we're proud of

Given the unreliability of non-invasive EEG headsets, our results are quite good

What we learned

Non-invasive EEG headsets produce really noisy outputs (A LOT of debugging and fixing stuff)
Common practices in the non-invasive space including SSVEP

What's next for Neuro North

Try with a better headset
- more than 8 electrodes
Experiment with a wider range of collection techniques (not just SSVEP) and signal processing techniques. This can align to different peoples strengths/limitations better.