Inspiration
Walking independently can be challenging for people with visual impairments, especially in environments with moving vehicles, obstacles, traffic signals, and crowded roads. We wanted to explore whether a regular walking stick could be enhanced with computer vision and sensors to provide useful, real-time assistance.
The idea behind VISION ASSIST came from the simple thought that technology should not only help people see information, but also help people navigate the world more safely and independently.
We wanted to build an affordable prototype that could identify important hazards and communicate them through simple voice and vibration alerts instead of requiring the user to look at a screen.
What it does
VISION ASSIST is an AI-assisted walking stick concept designed to support visually impaired users during navigation.
The system uses a camera and computer vision to identify objects in the user's surroundings, with a focus on vehicles such as cars, motorcycles, buses, and trucks. When a relevant vehicle is detected, the system provides a voice warning.
We are also developing an ultrasonic sensing module to detect nearby obstacles and provide additional alerts.
Our planned assistance features include:
🚗 Vehicle detection 🚶 Pedestrian and object detection 🚧 Nearby obstacle detection using ultrasonic sensing 🚦 Traffic signal recognition 🚸 Zebra-crossing detection 🔊 Voice alerts 📳 Vibration-based alerts 🆘 Emergency/SOS assistance 📍 GPS/GSM support for future versions
The goal is not to replace a traditional mobility aid, but to augment it with intelligent environmental awareness.
How we built it
We followed a modular approach so that each part could be developed and tested independently.
For the AI prototype, we used Python, OpenCV, and the Ultralytics YOLO object-detection model. A laptop camera provides the video input, OpenCV captures the frames, and YOLO processes them to identify objects.
For vehicle detection, we filter the detected classes to focus on:
car, motorcycle, bus, and truck.
We then connect the detected vehicle class to a corresponding voice message using text-to-speech.
The next hardware module uses an ESP32 and HC-SR04 ultrasonic sensor to measure the distance between the stick and nearby obstacles. This allows us to combine AI-based environmental recognition with physical distance sensing.
Our development flow is:
Camera ↓ OpenCV ↓ YOLO Object Detection ↓ Relevant Object Identified ↓ Decision ↓ Voice / Vibration Alert
The hardware side will follow:
HC-SR04 ↓ ESP32 ↓ Distance Calculation ↓ Obstacle Alert
Challenges we ran into
One of our biggest challenges was building the prototype from a beginner level while working with multiple technologies at the same time.
Setting up the Python environment and AI libraries required troubleshooting different Python versions and dependencies. We also had to understand how YOLO processes camera frames and how to extract the detected object classes.
Integrating voice alerts with real-time object detection was another challenge. The voice system worked independently first, so we tested each module separately before combining them.
We also had to consider hardware cost. Instead of depending on an expensive computing platform from the beginning, we chose a laptop-first development approach for the AI prototype and planned to move toward a compact ESP32-based hardware system.
Accomplishments that we're proud of
We are proud that we moved from an initial idea to a working AI prototype step by step.
Our current prototype successfully:
Captures live video through a camera Runs real-time YOLO object detection Identifies common objects Detects relevant vehicle classes Generates vehicle-specific voice warnings Provides a foundation for integrating ultrasonic obstacle detection
More importantly, we learned how to break a large assistive-technology idea into smaller modules and validate each component before combining them.
What we learned
This project taught us that building an AI product is not just about choosing a model. Integration, testing, hardware constraints, usability, and reliability are equally important.
We learned the basics of computer vision, object detection, Python libraries, camera processing, text-to-speech, and the process of connecting software intelligence with physical sensors.
We also learned the importance of designing around the user's needs. For a visually impaired user, an alert must be simple, timely, and understandable rather than overwhelming the user with unnecessary information.
Most importantly, we learned to build iteratively: start with one working feature, test it, identify problems, and then add the next feature.
What's next for VISION ASSIST
Our next goal is to move from the laptop-based AI prototype toward a more complete wearable walking-stick prototype.
We plan to integrate:
ESP32 + ultrasonic sensing for obstacle-distance detection Vibration feedback for immediate physical alerts Traffic light detection Zebra-crossing detection Tamil and English voice assistance SOS functionality GPS/GSM-based emergency assistance, where feasible
We also want to improve detection reliability in different lighting and outdoor conditions and test the prototype in realistic environments.
Our long-term vision is to make VISION ASSIST an affordable, practical assistive technology that helps visually impaired people navigate with greater confidence and independence.
Built With
- artificialintelligence
- assistivetechnology
- computervision
- embeddedsystems
- esp32
- hc-sr04
- iot
- machine-learning
- objectdetection
- opencv
- python
- pyttsx3
- realtimedetection
- smartmobility
- texttospeech
- ultralytics
- ultrasonicsensor
- voicealerts
- yolo