Birds of Consequence

• 

  1st Voice message - Bird imitation (Yellow bird), 2nd - Shot power / angle

• 

  Game execution

Inspiration

Unintentional use - What is more unintended than using WhatsApp voice messages to control Angry Birds? Definitely not what the founders imagined.

What it does

Voice messages are sent on WhatsApp via a Webhook through to a local server running Angry Birds (Python version). The 1 message imitates a bird, which is matched to 1 of 3 bird audios to select the character. A second audio is sent to determine the shot. The pitch of the message is used to calculate the angle of the shot whilst the length determines power.

How we built it

The program was built in 3 parts. Trilio, Flask and ngrok were used to communicate between Whatsapp and a local server on the host laptop. A Git repository of python files were used to take the .opus audio files and convert to .wav to select a character. The python scripts calculate the character (using PANN CNN14 to classify the bird imitation to a character) and the shot power. The game is then played on a version of Angry Birds found online running in Python.

Challenges we ran into

Many challenges were involved. Particularly hosting the ngrok server and being able to use WhatsApp to communicate to the local server. Due to a lack of compute power, we could not train a CNN locally, so we found pre-trained models online and attached a linear head in order to classify our sounds. This proved challenging and required out own sample set to be generated (using our own voices!). Furthermore developing the game mechanics to work with out inputs also took some attnention.

Accomplishments that we're proud of

We are proud of dominating the piggies! The process successfully shoots an Angry Bird Character at the piggies' defenses.

What we learned

We previously did not have experience with ngrok, Trilio and CNN implementation. We learnt that not all abilities in imitating bird sounds are created equal!

What's next for Birds of Consequence

More birds! More abilities and more levels!

Built With

• flask
• librosa
• ngrok
• pann/cnn14
• python
• twilio
• whatsapp