-
-
Soldering vibration motors
-
Testing the demo
-
Screenshot of the webcam from the demo
-
Screenshot of the webcam from the beta
Inspiration
Navigating a room is easy to take for granted. Speech assistants can tell a blind or low-vision person what’s in a room, but the last arm’s length of a reach, getting a hand to a cup or a door handle, is still guesswork. We wanted to close that gap: let words handle the coarse part, and let touch handle the precise part.
What it does
Glance is a voice-controlled assistive prototype. A head-worn camera sees the room. The user speaks to ask for an object or to have text read aloud. When their hand is within reach of the target, a glove on the right hand takes over. Vibration motors on the thumb, index, middle, ring and pinky fingers and the palm and back of your hand buzz to say which way to move. It is a prototype, not a safety device, and we make no safety claims.
How we built it
Vision: object detection finds the named target, and a tracker follows it while the hand moves. We find the hand, work out its position relative to the object and turn the difference into direction and distance.
Planner and speech: speech recognition turns the request into a plan, and a language-model planner chooses the target and steps.
Glove hardware: an ESP32 Feather drives two L293D motor-driver chips on a breadboard. Six vibration motors sit in a glove, with the wiring routed under a compression sleeve to a plug at the wrist. The laptop sends commands to the ESP32 over USB serial, and the firmware cuts the motors off if commands stop arriving.
Challenges we ran into
Getting vibration motors. We couldn’t get the coin motors in time, so we bought Minion-themed kids’ toothbrushes and took the vibration motors out of them. Hand tracking with a glove. Standard hand tracking relies on the contour of bare skin. With a glove on, that doesn’t work, so we built our own tracker based on color recognition. Measuring depth. Working out how far an object is from the hand was hard, so we built our own formula for it. Power and wiring. We debugged shared grounds, driver wiring and a lot of jumper wires on a breadboard, with no time for a custom circuit board.
Accomplishments that we're proud of
A working end-to-end loop from a spoken request to a vibration on the hand. Our own glove-friendly hand tracker and depth formula, built from scratch during the hackathon. Turning toothbrush motors into a wearable haptic glove on a tight deadline.
What we learned
Splitting the job works: speech handles the coarse navigation, and touch handles the last arm’s length. Hardware is mostly about the grounds, the supply and the pins. A shared ground fixes more problems than any code change. How to design for a user who can’t check the result by sight. Feedback has to be simple and immediate.
What's next for Glance
Add disambiguation so it can pick the right object when there are several similar ones. Move to a custom circuit board and a fully wireless glove, so the user isn’t tied to a bread board. Test it with blind and low-vision users, and measure how accurate and fast the guidance really is.
