DotByDot

Inspiration

Global braille literacy rates are at a historic low, with only 1 in 10 visually impaired individuals able to read braille. Existing Braille keyboard are prohibitively expensive, and also have steep learning curves to use. We were inspired to create a solution that tackles this literacy gap by combining intuitive design with accessible hardware. Our goal was to make braille education both obtainable and engaging for learners around the world.

What It Does

DotByDot is an adaptive web platform for learning braille through gamified lessons and tests. It features:

Accessibility-First Approach: Semantic HTML, ARIA attributes, and full keyboard navigability.
Global i18n Support: Easily switches to different languages for worldwide inclusivity.
AI-Powered Adaptation: Uses Google Gemini AI to adjust lesson difficulty in real time, tailoring a unique learning curve for each user.
Custom Hardware Integration: A low-cost braille keyboard built from tactile switches and PCBs, plus an LCD prototype to simulate refreshable braille cells.

How We Built It

Frontend: Created using Next.js and React, incorporating Tailwind CSS for rapid UI development and a strong focus on accessibility.
Backend & AI: Built on FastAPI with Python. Integrated Google Gemini AI for generating and adapting braille exercises based on user performance.
Database: Stored user progress and data in AWS DynamoDB.
Hardware: Designed a custom braille keyboard using affordable, durable switches and PCBs. Prototyped a refreshable braille cell concept with an LCD display.

Challenges We Ran Into

Accessibility: Ensuring a consistent, screen-reader-friendly experience across all lessons and pages. Ensuring the website was fully keyboard navigate-able and worked well with screen narrator systems was hard.
Hardware Integration: Calibrating the custom braille keyboard to interface seamlessly with the web platform. We faced a lot of last-minute challenges in interfacing the keyboard with the web platform, and it was rewarding to complete.
AI Adaptation: Fine-tuning the AI’s difficulty adjustment to accurately reflect user skill progression.
Localization: Managing multiple languages and ensuring lessons adapted properly for each locale.

Accomplishments That We’re Proud Of

True Accessibility: Successfully combining software and hardware solutions that address multiple accessibility barriers.
Low-Cost Hardware Prototype: Building a braille keyboard with refreshable cell concepts at a fraction of traditional costs. Utilizing a lot of the resources provided by Northwestern to create a hardware component was enriching and emphasized finding new solutions.
AI-Driven Personalization: Implementing a system where lesson difficulty changes in real time, keeping learners both motivated and challenged. We learned a lot in interacting with Gemini API in the backend to use its capabilities in content generation and other strengths.
Global Reach: Integrating i18n practices to make DotByDot useful for a broad, international audience.

What We Learned

Hardware-Software Synergy: Gained insight into bridging physical and digital design for accessible devices.
Adaptive Learning Algorithms: Explored how AI can dynamically tailor educational content to maintain engagement.
Inclusive Design Practices: Deepened our understanding of ARIA attributes, semantic HTML, and best practices for visually impaired users.
Localization at Scale: Learned to manage translation files and language detection in a complex, interactive web application.

What’s Next for DotByDot

Refined Braille Display: Evolve our LCD prototype into fully refreshable braille cells with open source modules.
Expanded AI Capabilities: Use more robust training data to enhance lesson variety and accuracy.
Community Contributions: Encourage developers to extend or customize our open source hardware for specialized needs.
Broader Language Support: Continue adding new languages and advanced braille features to serve an even wider global audience.