Inspiration
Braille is essential for many visually impaired users, but caregivers, teachers, volunteers, and accessibility workers may not always be able to read it quickly. We built BrailleVision AI to bridge that gap by using a phone camera to scan physical Braille and convert it into readable and spoken English.
What it does
BrailleVision AI lets a user point a mobile camera at physical Braille, capture an image, and receive an English translation with speech output. The backend detects Braille dot patterns from real camera images, groups dots into 2x3 Braille cells, decodes the cells into English, and returns an annotated image showing detected dots and cell regions. When the local computer vision engine is uncertain, the system can use an AI vision fallback to improve usability.
How we built it
We built a Flutter mobile app for the scanning experience and a FastAPI backend for image processing. The app uses the phone’s rear camera to capture Braille images, sends them to the backend, and speaks the translated text using text-to-speech.
The backend uses OpenCV to preprocess the image, enhance contrast, threshold dot-like regions, detect circular Braille dots, cluster them into rows and columns, group them into 2x3 Braille cells, and translate the detected dot patterns into English Grade 1 Braille. We also return an annotated image so users and judges can see the actual detected dots and cell boxes.
Challenges we ran into
The hardest part was not translating Braille symbols, but detecting physical Braille reliably from real camera images. Lighting, shadows, paper angle, blur, and dot size can change a lot between captures. We had to tune preprocessing, blob detection, contour filtering, row clustering, and cell grouping so the system could work on real physical Braille instead of only clean digital Braille characters.
Accomplishments that we're proud of
We built a working mobile-first prototype that captures real camera images, sends them to a live backend, detects physical Braille dots, groups them into Braille cells, translates the result into English, displays an annotated detection image, and speaks the translated text aloud. We are especially proud that the app includes both accessibility-focused interaction and a transparent computer vision pipeline.
What we learned
We learned that physical Braille recognition is a computer vision problem before it is a translation problem. The key is reliable dot detection, spacing estimation, and cell segmentation. We also learned how important accessibility-focused feedback is: users need large controls, speech output, and simple guidance while scanning.
What's next for BrailleVision AI
Next, we would improve perspective correction, add real-time frame-by-frame scanning, collect a larger dataset of embossed and handwritten Braille samples, train a dedicated dot detection model for poor lighting conditions, support more Grade 2 contractions, and add offline mobile inference for faster and more private use.
Log in or sign up for Devpost to join the conversation.