Inspiration
The app was inspired by how I, someone with ADHD, often zone out during conversations and lose track of where things have gone, or sometimes I zone out in lectures and get lost.
What it does
Using an EEG (mocked due to lack of hardware at event), the program measures how focused you are. When focus drops below a specified level, the microphone on the glasses or your laptop activates, and begins recording what is being said around you. When your focus returns, the audio is turned into text, and it is summarized by a large language model. Once the summary is produced, the text is displayed on a pair of Even G1 glasses, which contain a display, so that you may read what has occurred without being distracted or occupied by something like earbuds or needing to look down at your computer.
How we built it
I used the Scouterkit custom firmware for the glasses, and used Claude for writing the backend and frontend, using the m3-svelte library for UI.
Accomplishments that we're proud of
The app is fully functional end to end, and the speech to text system is able to consistently transcribe audio being said around the user to quickly show a summary of what has occurred while you are unfocused.
What we learned
Picking the correct speech to text model was challenging, with an initial choice for a streaming/realtime model being swapped for one that runs once when focus is resumed, improving performance. Additionally, managing the glasses and ensuring the connection is stable proved to be a challenge.
Built With
- llm
- parakeet
- sveltekit
- tts
