Inspiration: 1 in 5 people have dyslexia — the most common learning disability in the world. Yet most assistive technology requires pre-formatted digital content. Nothing works on the physical world. A dyslexic kid in a classroom can't use a screen reader on a whiteboard. An adult with dyslexia can't use an app to read a restaurant menu in real time. We wanted to build something that worked everywhere, on any text, with zero setup. That's where Lexia came from.
What it does: Lexia is a real-time wearable reading assistant for people with dyslexia. A camera mounted on a pair of glasses streams what the user sees to an on-device pipeline. When the user raises their open palm toward any text, Lexia detects the gesture using MediaPipe Hands, captures the frame, runs Tesseract OCR locally, reformats the text using dyslexia-friendly typography rules — OpenDyslexic font, 1.5x line spacing, single line isolation, high contrast — and reads it aloud in under 200ms. No cloud. No LLM. No internet. Fully offline. It also features word-by-word highlighting synced to the audio, multi-language OCR support for 100+ languages, and session analytics tracking reading speed and progress over time.
How we built it: We built Lexia as a fully local Python pipeline. OpenCV handles the camera feed. MediaPipe Hands detects the open palm gesture and measures a 1-second dwell time before triggering — preventing accidental reads. Tesseract OCR extracts text from the captured frame entirely on-device. A custom text reformatter applies dyslexia-friendly rules before passing the output to pyttsx3 for offline text-to-speech. The UI is built in Tkinter showing a live camera feed, palm detection indicator with countdown ring, state indicator, raw OCR output, and the reformatted dyslexia-friendly text panel with word-by-word highlighting. The wearable form factor uses a Raspberry Pi Zero 2W paired with an Arducam IMX708 camera module mounted on a glasses frame, streaming video over WiFi to the laptop.
Challenges we ran into: The biggest challenge was hardware. The FPC ribbon cable connecting the Arducam camera to the Pi Zero failed during the hackathon, forcing us to adapt the pipeline to run on a laptop webcam for the demo. We also had to solve the palm gesture dwell timer carefully — too sensitive and it triggers accidentally, too slow and it feels unresponsive. Getting the word-by-word TTS highlighting to sync accurately with pyttsx3's speech output required building a custom timing model since pyttsx3 doesn't expose word-level callbacks natively.
Accomplishments that we're proud of: We built a fully offline, zero-cloud assistive technology pipeline in under 24 hours. Every component — OCR, gesture detection, TTS, reformatting — runs locally with no API calls. The palm gesture trigger was a deliberate UX decision: a dyslexic kid in a classroom doesn't want to speak a wake word out loud in front of everyone. Raising a hand is natural, invisible, and something kids already do. We're proud that Lexia works on any physical text in any of 100+ languages, making it accessible to dyslexic people worldwide, not just English speakers.
What we learned: Hardware is humbling. Software you can debug — a broken ribbon cable at a hackathon is a different problem entirely. We learned to always have a software fallback ready. We also learned that the best assistive technology is invisible — the design of the gesture trigger came from thinking deeply about the social experience of being a dyslexic student, not just the technical problem.
What's next for Lexia: We want to get the wearable hardware fully working with the Pi Zero 2W and Arducam camera as originally designed. Beyond that — a lightweight companion app for parents and teachers to view session analytics, a custom fine-tuned OCR model for handwriting recognition, and exploring on-device inference on the Pi itself to make Lexia fully self-contained without needing a laptop. Long term, we want Lexia to be a $87 device that any school district in the world can afford to give to every dyslexic student.
Log in or sign up for Devpost to join the conversation.