gesturebot

• 

  Top view of gesturebot

• 

  Side view of gesturebot

Inspiration

Biomedical devices and assistive technology often leave out individuals who don't fit standard physical baselines. Most spatial tracking systems rely on ideal range-of-motion models, making them unreliable for people with hand tremors, nerve damage, physical limb differences, or even non-standard hand sizes. We wanted to build gesturebot to bridge this accessibility gap, proving that physical differences should never prevent someone from seamlessly interacting with and controlling modern robotics.

What it does

Standard gesture tech fails when hands aren't "normal." Our software uses a quick visual calibration process—capturing brief photo or video data—to map and normalize any user's unique hand motion in real time. By transforming non-standard or altered hand coordinates into a standardized control input, gesturebot allows users with tremors, nerve injuries, or non-baseline hand sizes to operate custom hardware and robotics smoothly, accurately, and effortlessly.

How we built it:

• Built using an ESP32 development board mounted on a solderless breadboard, powered by a 4-slot AA battery pack, paired with an audio speaker module for real-time auditory feedback, wired passives, and a custom chassis layout. • Developed software pipelines using Python, C/C++, and VSCode. We created custom algorithm layers to handle real-time spatial normalization, smoothing jitter from tremors, and re-scaling spatial bounds based on calibration snapshots. • Programmed the ESP32 (via C++/Arduino framework) to receive normalized spatial tracking commands over wireless communication (Wi-Fi/Bluetooth) to trigger hardware actions and audio cues.

Challenges we ran into

• Managing stable power delivery from the AA battery pack to both the ESP32 microcontroller and speaker module without experiencing voltage drops or signal noise across prototyping breadboard connections.

Accomplishments that we're proud of

• We created an accessible, software-driven calibration routine that works with basic camera hardware without needing expensive specialized sensors.

What we learned

What's next for gesturebot

Built With

• 4-slot-aa-battery-holder
• breadbroad
• c/c++
• esp32-development-board
• jumper-wires
• python
• speaker-module