Inspiration
Millions of blind students learn, read, and complete assignments in Braille every day. Yet once they leave the classroom, many parents are unable to read their homework, and many teachers lack the specialized training or expensive equipment required to quickly assess Braille work. As a result, students often wait days or even weeks for feedback, slowing learning and reducing opportunities for meaningful support at home.
We asked a simple question:
What if an ordinary smartphone could read Braille?
LUMEN was inspired by the idea that AI should not only make information easier to create—it should make education more accessible. Instead of building another AI tutor, we focused on removing one of the biggest barriers in Braille education: the inability of most people to read and assess Braille without specialized tools.
What it does
LUMEN teaches an ordinary smartphone to read Braille.
Using guided multi-angle image capture, LUMEN reconstructs the embossed Braille dots that are nearly invisible in a normal photograph. The reconstructed Braille is translated into digital text and enhanced with GPT-5.6 to provide contextual understanding and educational feedback.
LUMEN can:
- Capture Braille using only a smartphone camera
- Reconstruct embossed Braille through multi-angle image processing
- Translate Braille into readable text
- Compare assignments against instructor answer keys
- Generate explainable grading and feedback
- Produce tactile correction maps showing which Braille dots should be changed
- Help parents read and understand their child's homework
- Help teachers assess Braille assignments without specialist equipment
The result is faster feedback, more inclusive learning, and greater independence for blind students.
How we built it
LUMEN was developed using Codex with GPT-5.6 as an engineering partner throughout the entire development process.
We used Codex to explore alternative computer vision approaches, design the software architecture, generate modular implementations, review engineering decisions, and iteratively refine the system.
The final pipeline combines several components:
- Guided smartphone image capture
- Page isolation and perspective correction
- Multi-image Braille reconstruction
- Grid-based Braille dot detection
- Deterministic Braille decoding
- GPT-5.6 contextual reasoning
- AI-generated educational feedback and grading
Rather than relying on GPT for image recognition, we intentionally separated deterministic computer vision from language reasoning. The vision pipeline performs reconstruction and decoding, while GPT-5.6 provides contextual interpretation, explanations, grading assistance, and personalized educational feedback.
Challenges we ran into
The biggest challenge was reconstructing tiny embossed Braille dots using only a standard smartphone camera. Unlike printed text, Braille depends on subtle differences in light and shadow that are difficult to capture consistently in a single image.
We also had to carefully determine where traditional computer vision should end and where GPT-5.6 should begin. Instead of using AI as a black box, we designed a hybrid pipeline where deterministic algorithms handle reconstruction while GPT-5.6 focuses on contextual reasoning and educational assistance.
Another challenge was balancing technical depth with accessibility. The underlying algorithms are complex, but the user experience needed to remain simple enough for parents and teachers with no technical background.
Accomplishments that we're proud of
We're proud of creating a solution that removes one of the largest accessibility barriers in Braille education without requiring specialized hardware.
Highlights include:
- Teaching an ordinary smartphone to read Braille
- Building a modular multi-image Braille reconstruction pipeline
- Combining deterministic vision with GPT-5.6 reasoning
- Creating explainable grading and educational feedback
- Designing a workflow that empowers both teachers and parents
- Demonstrating how Codex can accelerate complex software engineering beyond simple code generation
Most importantly, we're proud that LUMEN focuses on enabling inclusion rather than replacing educators.
What we learned
Building LUMEN reinforced that meaningful AI products begin with understanding people rather than technology.
We learned that accessibility challenges often require combining multiple disciplines—including computer vision, human-computer interaction, and large language models—rather than relying on a single AI solution.
Working with Codex also changed how we approach software development. Instead of using AI simply to generate code, we used it as an engineering collaborator for architecture, design reviews, implementation planning, and iterative refinement.
What's next for LUMEN
Our immediate goal is to improve reconstruction accuracy across different lighting conditions, paper types, and smartphone cameras while expanding support for additional Braille standards and languages.
Future work includes:
- Real-time guided capture
- Offline on-device inference
- Multilingual Braille translation
- Teacher collaboration tools
- Student learning analytics
- Integration with learning management systems
- Personalized GPT-powered tutoring based on graded assignments
Long term, we envision LUMEN becoming the accessibility infrastructure that enables every parent, teacher, and student to participate fully in Braille education—using nothing more than the smartphone already in their pocket.
Log in or sign up for Devpost to join the conversation.