SAHARA

SAHARA (Smart Assistive Help and Accessibility For Reliable Autonomy) is a unified web application designed to assist individuals with visual, hearing, and physical impairments

Comment

Inspiration

Many digital accessibility tools on the market share the same fatal flaws: they require expensive subscriptions, mandate constant high-speed internet connections, and send sensitive personal data (like camera feeds and location) to remote cloud servers. We wanted to build a solution that respects user privacy and works anywhere—whether you are in a subway station with zero service or navigating a busy street. SAHARA was inspired by the idea that true accessible independence should be free, instantaneous, and entirely private.

What it does

SAHARA is a comprehensive, 100% offline-capable accessibility suite built into a single web application. It features six core modules: AI Sign Language Tutor: Uses real-time hand tracking through the device camera to evaluate and teach American Sign Language (ASL). AI Object Vision: Identifies objects in the user's environment using local machine learning. Live Captions: Provides large, readable text transcriptions of spoken conversations in real-time. Read Aloud: Converts any pasted text into natural speech. Magnifier: Transforms the device camera into a digital magnifying glass with zoom and color-inversion for low-vision users. Safety Check-in: Generates a rapid emergency message containing the user's exact geolocation to share with trusted contacts. Additionally, SAHARA features an "Ask Sahara" command bar that uses local intent routing to navigate users to the right tool using natural language.

How we built it

SAHARA was engineered as a fully client-side Single Page Application (SPA) using HTML, CSS, and Vanilla JavaScript. We made a strict architectural decision to avoid external APIs and backends. We leveraged the ml5.js library to run pre-trained machine learning models (MobileNet for object recognition and Handpose for ASL tracking) directly inside the user's browser. We utilized native browser APIs, including the Web Speech API (SpeechRecognition and SpeechSynthesis), the Geolocation API, and the MediaDevices API to power the core functionalities. The UI was designed with a dynamic "Living Desert Sky" theme that shifts based on the user's local time, alongside a dedicated accessibility panel for high-contrast and reduced motion preferences.

Challenges we ran into

Shifting from a cloud-based architecture to a strictly local, in-browser model presented significant hurdles. Our biggest challenge was managing the ml5.js video bindings. Initially, the object recognition model would freeze on an "Analyzing..." state because the browser's security protocols blocked the video feed processing. We had to rewrite the initialization logic to explicitly pass the DOM video element to the classifier only after the camera stream was fully resolved. Additionally, calculating the spatial logic for the ASL tutor—using Euclidean distance formulas to determine if a hand was open or closed based on digital landmarks—required precise fine-tuning to prevent false positives.

Accomplishments that we're proud of

We are incredibly proud that we managed to get a complex, real-time Computer Vision model (Handpose) to evaluate sign language locally without dropping frame rates or crashing the browser thread. Furthermore, we succeeded in our primary goal: building a premium, highly polished user interface that doesn't feel like a sterile utility app, but rather a welcoming, personalized companion. Proving that robust AI accessibility tools can be built without relying on costly server infrastructure is a massive win.

What we learned

This project fundamentally changed how we view front-end development. We learned just how powerful modern native browser APIs have become—specifically that you can perform real-time speech-to-text, text-to-speech, and machine learning inference entirely on the client side. We also learned the importance of "graceful degradation"; handling camera permissions and API unsupported errors without breaking the rest of the application's flow.

What's next for SAHARA

Currently, our ASL tutor evaluates basic signs like "Hello" and "Yes". Our immediate next step is expanding the local dictionary to support a wider array of phrases and dynamic movements. We also plan to convert SAHARA into a full Progressive Web App (PWA) complete with a service worker, allowing users to install the app natively on their home screens for even faster access to these life-saving tools.

Built With

  • canvas-apimobilenet
  • computer-vision
  • css3
  • geolocation-api
  • handpose
  • html5
  • javascript
  • machine-learningaccessibility
  • mediadevices-api
  • ml5.js
  • offline-first
  • privacy
  • tensorflow.jsweb-speech-api
  • ui-design
Share this project:

Updates