-
-
Static image prediction (takes 2-3 minutes, CPU)
-
Live feed detection(requires GPU,but lighting sensitive)
-
Architecture planned
Inspiration
Braille is a vital tool for literacy and independence, but for educators, parents, and peers who don't know it, a massive communication barrier exists. We wanted to build a bridge between the tactile world of Braille and the digital world. By creating BrailleX, we aimed to make Braille instantly accessible and understandable to anyone with a smartphone or laptop, breaking down inclusivity barriers in real-time.
What it does
BrailleX is an AI-powered accessibility platform that translates Braille into clear, readable English. Users can upload a static image of Braille text or launch a live video feed to scan physical surfaces on the go. To ensure absolute accuracy, the platform runs the translated text through an automatic correction layer to fix syntax errors. Finally, an integrated text-to-speech engine speaks the translated English aloud, making it completely multi-modal.
How we built it
- Frontend: Developed a sleek UI using React, TypeScript, and Vite to handle responsive camera feeds and seamless file uploads across mobile and desktop devices.
- Backend & AI Pipeline: Powered by Python, we built a custom processing pipeline to handle computer vision tasks and model inference.
- NLP & Speech: Implemented an auto-correct language model to clean up translation anomalies alongside a Text-to-Speech engine for instant vocal feedback.
Deep Learning & Dataset Details
Our character recognition engine is trained on the shanks0465/braille-character-dataset from Kaggle.
- Data Foundation: Started with 1,560 original images (60 per letter × 26 letters) at 28×28 pixels in grayscale.
- Augmentation Pipeline: To survive real-world deployment conditions, we augmented the dataset to 12,000+ images using photo-realistic augmentations (blur, noise, shadow, perspective distortions, and rotation).
- Data Split: Evaluated using a strict 70% train / 15% val / 15% test split, hitting a peak validation accuracy of 89.32% at epoch 46.
Model Architecture
We designed a lightweight, highly efficient Convolutional Neural Network (CNN) tailored for micro-deployments:
- Input Layer:
(28, 28, 1) - Block 1:
Conv2D(32) → BN → ReLU → Conv2D(32) → BN → ReLU → MaxPool → Dropout(0.30) - Block 2:
Conv2D(64) → BN → ReLU → Conv2D(64) → BN → ReLU → MaxPool → Dropout(0.30) - Block 3:
Conv2D(128) → BN → ReLU → GlobalAveragePooling → Dropout(0.45) - Classification Head:
Dense(256) → Dropout(0.35) → Dense(26, softmax)
Footprint Optimization:
With a total parameter count between 120,000 and 180,000, our compiled .keras model is only ~400–600 KB. When quantized to TFLite, the footprint drops to just 150–200 KB, making it ultra-efficient for edge computing and instant web-browser loading.
Challenges we faced
Handling live video feeds came with major hurdles, particularly with shadows, varying lighting conditions, and camera angles skewing the Braille dots. We overcame this by implementing our heavy photo-realistic augmentation pipeline during training so the model could generalize to imperfect real-world frames. Optimizing the pipeline to run image processing, model inference, text auto-correct, and audio generation in near real-time without lagging the live feed was another massive technical hurdle we managed to overcome.
Accomplishments that we're proud of
We are incredibly proud of building a fully functional, end-to-end accessibility pipeline during the hackathon. Achieving low-latency translation on a live camera feed while simultaneously correcting grammar and speaking the text aloud felt like magic when it finally came together.
What we learned
We learned a tremendous amount about computer vision optimization, quantization for edge devices, and the nuances of Braille cell structures. More importantly, this project deepened our understanding of inclusive design and how heavily assistive technology relies on minimizing user friction.
What's next for BrailleX
We plan to expand BrailleX to support two-way translation (English text back into digital Braille cells) and optimize our TFLite models to run entirely client-side in the browser or via standalone mobile apps. We also hope to partner with educational institutions to test BrailleX as a learning tool for teachers and parents of visually impaired children.
