Guiden: Multimodal Mobile AI Guide for the Visually Impaired

Team

Samuel Philip
Anish Ganapathi
Nishwa Prajapati

slide-1

What is Guiden?

Guiden is a fully voice-controlled mobile AI assistant designed to help blind and visually impaired users understand and navigate the physical world safely, independently, and hands-free.

Unlike traditional accessibility tools that rely on screen readers or visual interfaces, Guiden is built voice-first, allowing users to interact with the entire app using natural speech.

The Problem

For blind individuals, everyday activities like walking through a hallway, identifying objects, or knowing whether a room is well-lit can be challenging and unsafe.

Most navigation and utility apps assume visual interaction, leaving blind users dependent on external assistance. We wanted to build a solution that provides real-time environmental awareness using only a smartphone.

Our Solution

Guiden transforms a smartphone into an intelligent guide by combining on-device perception, multimodal AI reasoning, and natural voice interaction.

The app includes three core voice-activated modules:

Navigation Assist

Users can say "Navigate Assist" to activate hands-free path guidance.
Using on-device object detection, Guiden identifies obstacles in real time and provides continuous audio instructions to help users move safely without touching the screen.

Light Frequency Mode

By saying "Navigate Light Frequency", users can understand room lighting through sound.
Guiden converts ambient light intensity into varying audio frequencies, allowing blind users to sense brightness levels without spoken feedback.

Voice Assist (Object and Product Identification)

With "Navigate Voice Assist" and a simple "Capture" command, Guiden analyzes objects in front of the camera.
It identifies items and explains their purpose, helping users recognize products, everyday objects, and surroundings.

Technology Stack

On-device YOLO object detection for real-time obstacle and object recognition
Gemini Multimodal Vision AI for contextual understanding and reasoning
ElevenLabs for natural, human-like voice feedback
A custom AI pipeline balancing low-latency on-device inference with powerful cloud intelligence

Impact

Guiden is designed to promote accessibility, independence, and dignity.

By removing the need for visual interaction, it empowers blind users to navigate spaces, understand environments, and identify objects confidently using only their voice.

Built at Stevens QuackHacks 2026, Guiden demonstrates how AI can be used responsibly to create inclusive, human-centered technology.