Synthly

Intro

Synthly is a new way of using modern technology to create music using hand gestures. Enabling a new world of creativity.

Inspiration

Being a DJ has crossed our minds countless times and upon learning about the tech capable of hand gesture detection, we were all in for building a tool for professionals and amateurs alike. Inspired by the natural hand gestures that humans use to communicate with each other, with Synthly, musicians can now make use of a more natural way to formulate unique sounding music without the limitations of physical instruments. Simple yet powerful.

How we build it

We build Synthly using python and the OpenCV library. The hand detection technique utilizes an computational geometry algorithm called "convex hull" to draw a polygon using the outer most points on the point cloud of the hand. By detecting the number of valleys in the shape of the hand, we can then detect the number of fingers that are being held by the user.

Challenges we ran into

The main challenge that we ran into was the issue of applying the convex hull algorithm. This algorithm detects the number of fingers by counting the number of concaves between their fingers, which in turn only allows the user to play one of 5 notes; one per finger. This limited us in the design of Synthly but also empowered us to become more creative with fewer tools.

What we learned

We learned how the convex hull algorithm works and can be applied to the real world.

What's Next

Synthly is the first of many products that use modern technology to help artists and create and express themselves. We believe that the next step for Synthly would be enabling more notes playback by using a more powerful algorithm or machine learning to understand more hand gestures. Along with this, we want to be able to detect two hands at once so a user is able to play multiple instruments.

Built With

• python
• opencv
• numpy