Inspiration
Navigating the world safely and picking up on social cues can be really challenging for visually impaired people. We wanted to create something that could help—not just a mobility aid, but a companion that gives both environmental awareness and emotional context. The idea was simple: empower blind users to move confidently, engage socially, and feel safer, without relying solely on a guide dog or cane.
What It Does
B.E.A.D is a small wearable device powered by an ESP32. Here’s what it can do:
It uses a tiny camera to see the world around the user.
It runs two CNN models in real-time:
One (Roboflow-trained, 98% accurate) detects objects, doorways, and faces.
Another (OpenCV-trained, 99% accurate) reads facial emotions.
It communicates everything through haptic feedback using a small buzzer—so users can “feel” what’s happening around them.
Basically, it helps users understand their surroundings and the people in them, potentially replacing or complementing guide dogs or canes.
How We Built It
We combined hardware, software, and AI to bring B.E.A.D to life:
Hardware: ESP32, camera module, buzzer, and battery pack.
Software:
CNN model 1 for object, doorway, and face detection.
CNN model 2 for emotion recognition.
Code written in Arduino/MicroPython to capture the camera feed, run inference, and trigger the buzzer.
Workflow: Camera captures a frame → objects and faces are detected → emotions are recognized → buzzer provides feedback.
Challenges We Faced
Running two CNN models on a tiny ESP32 was tricky—memory and processing power are limited.
Getting real-time feedback without delays took a lot of optimization.
Designing buzzer patterns that are clear and distinguishable for objects, doorways, and emotions required careful testing.
Finding the balance between model size and accuracy for edge deployment was another hurdle.
What We’re Proud Of
Both CNN models now run on the ESP32, almost in real-time.
We hit high accuracy: 98% for object/door/face detection, 99% for emotion recognition.
Built a fully functional wearable prototype that gives users environmental and emotional awareness.
Developed intuitive haptic patterns that make sense without being overwhelming.
Showed that B.E.A.D could replace or supplement guide dogs or canes, improving mobility and social interaction.
What We Learned
Edge ML is powerful, but you have to optimize carefully.
Haptic feedback is surprisingly nuanced—you have to test it in real life to get it right.
Combining emotional awareness with environmental sensing creates a much richer, holistic assistive experience.
Real-world testing is essential to make everything intuitive and responsive.
What’s Next
Detect even more objects and obstacles for fuller environmental coverage.
Add voice feedback alongside haptics for a richer experience.
Improve battery life, comfort, and wearability.
Create a companion mobile app for logging events and notifying caregivers.
Explore LLM integration for interpreting social context and giving advice or guidance.
Log in or sign up for Devpost to join the conversation.