Inspiration
I wanted a hands-free way to control music on a PC. Spotify’s mobile air gestures inspired us to explore how an ordinary webcam could make desktop music controls accessible and fun.
What it does
GesturePilot uses a webcam to recognize hand poses and movements, then sends controls to Spotify on Windows. A closed fist toggles play and pause; pinch-and-drag gestures skip tracks ; a victory gesture switches Spotify between maximized and restored sizes; and an open-palm circle changes Windows master volume. Thumbs-up and thumbs-down send Spotify’s Like/Dislike Song shortcut.
How i built it
The app is written in Python. OpenCV captures webcam frames, MediaPipe detects hand gestures, and PyAutoGUI sends desktop input. Gesture mappings and tuning values are kept in gesture_config.json.
Challenges i ran into
A gesture can be misread when poses overlap or the hand moves between actions. We added neutral states, short holds, smoothing and one-action-per-pinch behavior to reduce accidental or repeated controls. Camera position, lighting and hand orientation still affect recognition.
What i learned
I learned how to connect real-time hand tracking to desktop controls, and how much careful gesture separation and feedback matter for a usable hands-free interface.
What’s next
I’d like to improve calibration and reliability across different webcams, lighting conditions and users.
Log in or sign up for Devpost to join the conversation.