MaestroBOT

Inspiration

Being able to create the equivalent of an orchestral conductor with respect to the electronic music applications.

What it does

We have 6 gestures encoding 6 instructions to the vlc-player.

• hand-tracking.py — captures webcam video, recognizes hand gestures, and prints gesture info.

• music_controller.py — launches hand-tracking.py, parses its printed gestures, and controls an MP3 using python-vlc; it also provides a small Tkinter GUI.

• Open Hand: start playback (or unpause if paused)

• Closed Fist: pause playback

• One Finger pointed: raise volume

• Two Fingers: lower volume

• Three Fingers: Increase playback speed

• Four Fingers: Decrease playback speed

The gestures will also each change the stereo image from left to right.

How we built it

REQUIRED INSTALLATION:

py -m pip install opencv-python mediapipe
py -m pip install python-vlc

Also required: cv2, vlc

Challenges we ran into

Ensuring compatibility across different OSes, allowing a wide variety of possible gestural musical modifications

Accomplishments that we're proud of

Getting a bare MVP version of gesture controlled audio parameters.

What we learned

How audio processing can be performed in Python, as well as using machine learning for gestural detection to this end.

What's next for MaestroBOT

The ability to modulate more audio parameters with more gestures could be supported, as well as better cross-OS compatibility.

Built With

• mediapipe
• open-cv
• python
• vlc