Inspiration
Navigating the world without sight is a daily challenge, and existing assistive technologies often fall short due to high costs, limited functionality, or a lack of real-time adaptability. Our goal was to develop an affordable, AI-powered system that enhances spatial awareness for visually impaired individuals, empowering them to navigate their surroundings safely and independently. By combining real-time computer vision with proximity sensing, we aim to provide a comprehensive solution that improves mobility, safety, and confidence.
What It Does
Spidey Sense is a smart, wearable navigation assistant that leverages AI-driven computer vision and proximity detection to help visually impaired users avoid obstacles and identify objects in real time.
Key Features
• AI Object Detection & Recognition – Using OpenCV and YOLO on an ESP32-CAM, the system detects and identifies objects in real time, providing relevant audio feedback.
• Proximity Alerts – Ultrasonic sensors measure distances to obstacles, triggering a beeping system to warn users of potential collisions.
• Multi-Modal Feedback System – Voice-guided notifications announce detected objects, while variable-frequency beeps indicate obstacle proximity, ensuring intuitive and immediate awareness.
• Wireless Connectivity & Edge Processing – The ESP32-CAM transmits images over WiFi to a laptop running a Python-based AI model, which processes data and sends audio feedback to the user in real time.
Build
Hardware:
ESP32-CAM for real-time image capture and processing.
Arduino Uno for controlling ultrasonic sensors and audio output.
Ultrasonic Sensors for distance measurement.
Speaker Module for relaying voice-guided assistance.
Software:
Python with OpenCV & YOLOv3 for AI-driven object detection.
C++ (Arduino) for sensor control and audio feedback processing.
WiFi Communication between ESP32-CAM and the laptop for efficient AI inference.
Challenges
• Efficient Image Transmission – Optimizing the ESP32-CAM's WiFi streaming to ensure smooth video feed without excessive latency.
• AI Model Optimization – Choosing a YOLO model to detect relevant objects while keeping processing lightweight.
• Sensor Integration – Synchronizing ultrasonic sensor data with AI-based object detection to provide accurate feedback.
Accomplishments
• Successfully integrating real-time object detection and proximity sensing into a single assistive device.
• Achieving low-latency WiFi communication between the ESP32-CAM and the laptop, enabling real-time feedback.
• Creating an affordable alternative to expensive assistive technologies, potentially making navigation aids more accessible to visually impaired individuals worldwide.
• Completing a working prototype within the constraints of a 24-hour hackathon!
The Future of Spidey Sense
We envision Spidey Sense evolving into a robust, real-world solution with the following improvements:
• Upgrading to a high-resolution camera with a wider FOV for improved object detection.
• Replacing ultrasonic sensors with more accurate sensors such as LiDAR for more precise depth perception.
• Enhancing AI models to recognize a broader range of objects and provide contextual information.
• Integrating GPS and voice assistant support to enable location-aware guidance and navigation assistance.
• Miniaturizing the hardware to develop a lightweight, wearable form factor such as smart glasses or a compact belt-mounted device.
By continuously refining Spidey Sense, we aim to create an affordable, real-time assistive navigation system that can significantly enhance mobility and independence for millions of visually impaired individuals worldwide.
Log in or sign up for Devpost to join the conversation.