I started working on these filters when I realised there are very few musical theatre games out there. The one's that are either trivia based or aimed at children for singalongs. I wanted to make games that my friends and myself would enjoy. COVID-19 Shut down all theatres and live performance spaces, we were all out of work and we were also very demoralised. I wanted to bring joy to performers and musical theatre fans everywhere. By making games that I would enjoy. That the enjoyment is in the experience, not in whether you win or lose, where their are no rules you can't break, where you share your excitement in knowing the words to a song, but also be willing to laugh at yourself and share your experience with your friends when you don't. I realised that the analytics side is quite important and one I downplayed the value of while making Augmented Reality filters. I realised that the data showed what musicals people enjoyed the most, what users were most popular and I could actually track how far it reached online. This is all data that could be used to help producers and theatres realise that when they open, there are fans out there willing to pay too enjoy the event and also, what shows and performers are currently popular among society.

What it does

These are a set of musical theatre instagram filters, they're created for fans to test their musical theatre knowledge and sing along to their favourite show tunes. The Title of the show appears on the user's forehead when the camera on their mobile device recognises a face, the filter then randomises through the show's musical numbers when pressing record before stopping, prompting the user to sing along. The recording is then posted on instagram, but the video can then be saved and uploaded to all social media platforms like Facebook, Youtube, TikTok, Twitter and LinkedIn.

How I built it

I have an arts degree, not a computer or engineering one. Everything I learnt about AR is from youtube tutorials, which I think is a testament of how using a visual patch layout opposed to scripting is easily accessible. The functionality of the patch is linking 1 function to the next: Find a face>Show the title>Press record>Delay randomizing for x seconds>Randomize through the assets or x seconds>Stop. The whole time this occurs it is told to track the user's face (how to identify a face is a separate patch altogether). The graphic assets are in 2 categories: The Title and Slides (The musical numbers). Upon recognising the face the filter will show the Title, and upon pressing record the Slides will randomise then stop. It sounds simple and to do a degree if you're a visual learner, it is. But the back end of having working Augmented Reality and facial recognition is a little more complex.

Challenges I ran into

Spark AR Studio literally released an update 2 days ago that broke my test links on android devices and I had to contact them at 10pm to troubleshoot. So using software that regularly updates and changes caused me to lose a bit of sleep over the last couple days. Luckily Spark AR Studio has the tech support of the Facebook and Instagram teams.

Accomplishments that I'm proud of

About 2 years ago I decided to learn game design, I taught myself C#, Unity, SparkAR, music editing, graphic design and started my own game design company. I published games on The Apple app store (where my one game made it to the number 1 spot in its category) as well as the Google Playstore. My Instagram filter profile currently has 24 million impressions and 2.7 million uses, which is insane for a self-taught theatre major on a 2012 Macbook.

What I learned

Not to be afraid of technology or to learn a new skill. People hear Augmented Reality or coding and immediately are intimidated. I work in theatre and it seems like a lot of the artists I work with have the mentality of "I'm creative, not technical" and avoid new technologies, when actually if they embraced it more people could be exposed to their work.

What's next for Augmented Reality Musical Theatre Karaoke Filters

South Africa has a National Arts Festival every year in June, due to COVID-19 closing all possibilities of a live theatre experience they have opted to go digital. I will be teaching a workshop on AR filter design and talking about how to integrate the arts into the modern world through new technologies available.

Built With

Share this project: