Indonesia is a beautiful country with thousands of cultures deeply rooted in its history. Yet sadly, according to a survey conducted by the Indonesian Ministry of Education and Culture, only 0.31% of Indonesian participate in any traditional theatre or "pedalangan" as we call it . One of Indonesia's oldest theatrical performances is Wayang which was originally found in the cultures of Java, Indonesia. It involves telling a dramatic story through shadows thrown by puppets.
We were inspired to bring this traditional experience to live inside an AR World Effect. So now with Wayang AR Effect, people who have never seen Wayang in real life can now experience it right from their phone! We hope this can inspire more people to learn not only about wayang but thousands of Indonesia's forgotten culture and heritage.
What it does
This AR effect will spawn Wayang, a traditional form of puppet theatre play originally found in the cultures of Java, Indonesia, on a plane in front of your camera. It also spawns a wall and a candle that will project the wayang's shadow onto the wall. You can tap the screen to turn on/off the light. We design it to mimic the experience of how it was done traditionally. Traditionally, the wayang gets upscaled in size, and through the shadow, a puppeteer tells its story.
How I built it
I used a 3D model of Wayang Golek Arjuna by The Seweryn Udziela Ethnographic Museum in Kraków & RDW MIC, Małopolska’s Virtual Museums project, and I reduce the amount of vertices in order to make it fit into the Instagram's size requirement. Next, I combine the 3D wayang model with a simple wall and candle. I then learned how to create texture and photoshopped a shadow of the wayang's model to both sides of the wall. Lastly, I stitch the model, a traditional gamelan sound, and on/off ambient & directional light patch on SparkAR Studio.
Challenges I ran into
I have 0 experience on SparkAR and Blender. Learning how to use both of these tools is quite a steep learning curve for me. Luckily, SparkAR and Blender have a lot of online tutorials and documentation.
Accomplishments that I'm proud of and what I learned
I'm really proud that I learned a lot not only about SparkAR but also Blender and Photoshop during the making of this submission. The knowledge that I gained in this competition will surely benefit me in the future.
What's next for Wayang
This submission is not what we initially planned. Our original plan was having a full wayang theatre sketch complete with a moving wayang, music, and a storyteller. So, that is the next step of this Wayang AR Effect. Hopefully, we can expand to not only Wayang Arjuna, but dozens of other traditional wayang play.
And finally, we can expand to not only wayang but also traditional Indonesian dances. We imagined that this is possible because we have been doing some experiments extracting keypoints using OpenPOSE on traditional Indonesian dances. Our current problem is in integrating the movement of the dance with a 3D model.
Try it out on Instagram: https://www.instagram.com/a/r/?effect_id=2252247328417723
GitHub link: https://github.com/ilhamfp/wayang
Every song, 3d model, and footage is under either free use or CC Attribution licenses.
 Indonesian Ministry of Education and Culture. (2016). Analisis Partisipasi Kebudayaan. Data and Statistics Center for Education and Culture. URL: http://publikasi.data.kemdikbud.go.id/uploadDir/isi_6B964F97-8196-45DA-95B7-715095411517_.pdf
 Wayang Golek Arjuna 3D Model by The Seweryn Udziela Ethnographic Museum in Kraków and RDW MIC, Małopolska’s Virtual Museums project. URL: https://sketchfab.com/3d-models/arjuna-wayang-golek-theatre-puppet-3043564cc38b49aa8b08a61ce68dd61a
 Candle 3D model by frostcorestudios. URL: https://free3d.com/3d-model/candle-717529.html