GuideSense

logo
screen before connecting camera
Session Timeline
Session Configuration
Viewable Example

Inspiration Imagine navigating your daily routine blindfolded—simple tasks become daunting, and the world feels unpredictable. This is the reality for approximately 285 million individuals living with visual impairments. While white canes help with ground-level obstacles, they leave users vulnerable to chest- and head-height hazards. Furthermore, existing audio aids constantly shout repetitive labels, creating severe alert fatigue and masking essential ambient sounds. Inspired by their resilience, we developed "GuideSense," a smart wearable navigation aid designed to provide clear, fatigue-free spatial guidance and instant collision protection.

What it does GuideSense is an on-device ambient awareness system for visually impaired individuals. Using a camera and real-time computer vision, it detects obstacles, calculates metric distance, and identifies spatial direction (left, center, right). Instead of constant beeping, its intelligent fusion core stays quiet when the path is clear, speaks calm guidance once for mid-range obstacles ("Chair on your left"), and instantly triggers an urgent voice warning at an accelerated speech rate when an obstacle enters close range ("Stop. Person very close ahead").

How we built it We combined edge computer vision with a priority-gated decision engine running 100% locally on CPU:

Perception: OpenCV with a lightweight MobileNet-SSD model for real-time 10 Hz object detection. Spatial & Distance Estimation: Monocular bounding-box height geometry to estimate distance without external rangefinder sensors, mapped into 3-lane spatial coordinates (Left, Center, Right). Fusion & Safety Core: A custom state machine featuring 3-tick persistence gating to eliminate flicker, object priority hierarchy (e.g., people over furniture), distance-adaptive cooldowns to prevent repetitive alarms, and timing hysteresis for stable state transitions. Audio Pipeline: A non-blocking pyttsx3 text-to-speech worker that dynamically shifts speech rates (calm 145 WPM for guidance vs. rapid 190 WPM for urgent stops). Challenges we ran into Balancing noise suppression with emergency safety was our primary challenge. Filtering out momentary camera flicker without delaying urgent collision warnings required building a dual-path pipeline: informational announcements pass through strict persistence gates, while close-range hazards immediately bypass all suppression filters. Additionally, stabilizing monocular distance estimation from 2D bounding boxes and preventing blocking text-to-speech calls from lagging our 10 Hz computer-vision loop required careful thread serialization.

Accomplishments that we're proud of Solving alert fatigue by creating an intelligent decision engine that knows when not to speak. Delivering a complete, zero-cloud AI vision and voice pipeline that runs locally on CPU with zero latency and full user privacy. Implementing directional voice guidance that naturally shifts cadence and urgency based on physical proximity. Building a comprehensive automated test suite verifying safety gates, priority tie-breaking, and boundary hysteresis. What we learned Assistive technology is ultimately an ergonomics challenge. We learned that more data is not always better—knowing when to remain silent is just as critical as knowing when to alert. Decoupling our vision detection, priority fusion, and audio synthesis into separate, modular components allowed us to fine-tune safety thresholds without compromising real-time performance.

What's next for GuideSense Looking ahead, we envision packaging GuideSense into an ultra-compact, 3D-printed chest clip or harness for all-day wearable use. We plan to integrate binaural bone-conduction audio to provide 3D spatial directional sound while keeping the user's ears open to the environment. Additionally, porting model inference to dedicated low-power edge NPUs will extend battery life, transforming GuideSense into a dependable, everyday mobility companion.