I have been learning Bharatanatyam since I was six years old. Bharatanatyam is the oldest form of classical Indian dance, and it combines precise rhythmic movements with expressive gestures to convey both dance and drama. As a long-time dancer, I am familiar with the precision with which dance steps are mathematically arranged within the music. The concept of quantized steps got me thinking of how it can be applied to choreograph a Bharatanatyam dance to music, thus inspiring Natya*ML.
What it does
Natya*ML has two parts: music and dance. The music is entirely ML-generated, and the user can pick which sound they would like to hear (mridangam #1, mridangam #2, violin, or harp) and which key they would like their music to be in. For the dance portion, I've created a library of 8 steps. The user may choose to loop over any of the steps, essentially viewing each step over and over again. The other option is to auto-choreograph a dance. The steps are chosen to go along with the beats of the music. Regardless of which option is chosen by the user, the canvas displays a recording of each step along with the skeletal rendering.
How I built it
The music was constructed using Tone.js and MusicRNN from Magenta.js. The skeletal rendering of the dancer was created using PoseNet from ml5.js.
Challenges I ran into
Accomplishments that I'm proud of
I am immensely proud of how much I accomplished. Considering that I started the hackathon with no knowledge of Tone.js, Magenta.js, and ml5.js, and very minimal experience with HTML and CSS, I'm so proud that I was able to learn this much and create Natya*ML. I'm also really happy that I was able to combine my 11 years of dancing and 8 years of learning piano with programming to create Natya*ML. I never really considered how much art can come into play with programming.
What I learned
Nearly everything you see in Natya*ML is something I learned about during this hackathon. This has been an amazing learning experience for me.
What's next for Natya*ML
I want to expand upon the video library of steps. There are about 175 steps in Bharatanatyam. Due to time and resource constraints, I only included 8. However, given more resources and time, I would include more. Also, if I were given more resources, I would want to integrate BodyPix and PoseNet to create a rendering of a dancer on a cool background.