MORPH

Logo
Drone

Inspiration

Most interfaces make people adapt to buttons and menus. We wanted to explore the opposite: a machine that understands what someone points at and adapts its controls to what they want to do.

What it does

MORPH is a prototype for “point, confirm, adapt, act.” Its pointing system identifies a target and waits for a steady gesture before confirming it. The larger goal is to connect that choice to physical controls, a robotic arm, and laptop actions.

How we built it

We built the pointing logic in Python, a live hand-tracking viewer using OpenCV and MediaPipe, and a desktop agent with a safe test mode. Our Raspberry Pi runs the pointing-decision tests and captures images from a USB webcam. An Expo phone interface and ESP32 hardware controls are being developed alongside the physical build.

Challenges we ran into

Getting camera access and hand tracking working across different computers was harder than expected. We also had to design the software before all the robot parts and wiring were ready.

Accomplishments that we're proud of

MORPH’s pointing tests pass on the Raspberry Pi, and the live viewer can detect a pointing hand and highlight a target. We built the system so decisions must remain steady rather than triggering on a single accidental movement.

What we learned

A good physical interface needs more than gesture recognition. It also needs clear confirmation, reliable communication between devices, and safe behavior when something goes wrong.