NaviSight AI\
Inspiration For a person who is blind or visually impaired, everyday activities that many people take for granted—walking through an unfamiliar place, identifying an obstacle, or reading information on a medicine package—can become difficult or require assistance from others.
We wanted to explore a simple question:
What if a person could receive useful information about the world around them through audio, without needing to see it?
This inspired us to build NaviSight AI, an AI-powered assistive vision system designed to turn visual information into meaningful spoken guidance.
Instead of simply detecting an object and saying "chair detected," NaviSight AI aims to understand the situation and provide an actionable instruction such as:
"Chair ahead. Move left and walk three steps, then continue straight."
We also wanted the system to help with another everyday challenge: identifying medicines and understanding the information printed on their packaging.
What NaviSight AI Does NaviSight AI combines a camera, distance sensing, artificial intelligence, and audio feedback.
🧭 Smart Navigation
The camera observes the environment while sensors help estimate nearby obstacles.
The AI can identify objects such as:
- Chairs
- Tables
- Walls
- Doors
- People
- Vehicles
- Stairs
- Other obstacles
The system then analyzes the user's path and converts the visual information into spoken guidance.
For example:
"Obstacle ahead. Move to the left."
or:
"Chair ahead. Move left, walk three steps, then continue straight."
The goal is not just object detection, but context-aware guidance.
💊 Medicine Scanner
NaviSight AI also includes a medicine-assistance mode.
When a medicine package is placed in front of the camera, the system can use OCR and AI-based processing to extract information such as:
- Medicine name
- Strength
- Expiry information
- Usage information
- Warnings
- Commonly listed side effects
The information is then converted into speech through the user's headphones.
For example:
"Paracetamol 500 mg detected. This medicine is commonly used for fever and pain."
The system is designed as an information and safety assistant, not as a replacement for a doctor or pharmacist. When information cannot be identified confidently, the system communicates that uncertainty rather than pretending to know the answer.
How We Built It
The system is designed around four major components:
Camera + Distance Sensors
↓
AI Vision Engine
↓
┌────────┴─────────┐
↓ ↓
Navigation AI Medicine AI
↓ ↓
└────────┬─────────┘
↓
Voice Assistant
↓
Headphones
1. Vision
A camera captures the environment in front of the user.
Computer vision models analyze the scene and detect relevant objects.
2. Distance & Spatial Understanding
Distance information is combined with visual detection to estimate how close an obstacle is and whether it is blocking the user's path.
3. Context-Aware Decision Making
Instead of simply reporting detected objects, the system considers the object's position and the available path.
For example:
Object detected
↓
Distance calculated
↓
Is it blocking the path?
↓
Which direction is clearer?
↓
Generate navigation instruction
4. Audio Interface
The final information is converted into concise spoken instructions and delivered through headphones.
This allows the user to receive information without looking at a screen.
What We Learned
Building NaviSight AI helped us understand that assistive AI is much more than simply training an object-detection model.
We learned about:
- Computer vision and object detection
- OCR and text extraction
- Distance and spatial reasoning
- Context-aware AI
- Voice-based interfaces
- Real-time processing
- Sensor integration
- Human-centered design
- Designing AI systems that communicate uncertainty
One of our biggest lessons was that accuracy alone isn't enough.
A system can correctly identify a chair, but if it doesn't understand whether the chair is actually blocking the user's path, the detection isn't very useful.
This led us to focus on the pipeline:
See → Understand → Decide → Speak
Challenges We Faced
Real-Time Processing
The system needs to analyze the environment continuously while keeping the response fast enough to be useful.
Balancing model accuracy with processing speed became an important design challenge.
Navigation Decisions
Detecting an obstacle is relatively straightforward compared with deciding what the user should do next.
We had to think about:
- Where is the obstacle?
- How far away is it?
- Is the path blocked?
- Is the left side clear?
- Is the right side clear?
- How should the instruction be communicated?
Medicine Recognition
Medicine packaging can contain small text, different fonts, reflections, curved surfaces, and multiple pieces of information.
OCR can therefore produce incorrect results.
For this reason, NaviSight AI is designed to communicate uncertainty and encourage verification when information cannot be reliably identified.
Safety
Because the system is intended to assist people in real-world situations, we had to think carefully about what the AI should and should not say.
For navigation, the system should provide assistance rather than claim perfect environmental awareness.
For medicines, it should provide information from the detected packaging and trusted sources rather than making personalized medical decisions.
What Makes NaviSight AI Different
Many assistive-vision systems focus primarily on recognizing objects.
Our goal is to move one step further:
From "I see an object" to "I understand what the object means for the user."
For example:
Traditional object detection:
"Chair detected."
NaviSight AI:
"Chair ahead. Move left and continue walking."
This shift from recognition to actionable context is the core idea behind NaviSight AI.
Future Vision
We envision NaviSight AI becoming a lightweight wearable assistant that can help users navigate indoor and outdoor environments more independently.
Future versions could include:
- GPS and turn-by-turn navigation
- Stair and curb detection
- Crosswalk recognition
- Traffic and vehicle awareness
- Face recognition with privacy controls
- Currency and document recognition
- Multi-language voice assistance
- Offline/on-device AI processing
- Emergency detection and alerts
- Personalized voice interaction
Our long-term goal is simple:
Use AI to make visual information accessible through sound.
NaviSight AI is not designed to replace human assistance or professional medical advice. It is designed to provide an additional layer of information and guidance that can help make everyday environments more understandable and accessible.
Log in or sign up for Devpost to join the conversation.