We were inspired to make a filter that can help people make interesting choreographed dance videos utilizing an on-screen portal that dramatically alters the users appearance when interacted with. Many users love to make videos that make use of dramatic transitions and showcase their creativity through contrast, so we built an effect that can help them accomplish that on Reels. We took inspiration from the anime art style popular in online culture today, and hoped that users could make use of our segmentation and appearance modification to build videos that resonate with their interests.

What it does

The filter starts with an on-screen portal, which disappears when the user interacts with it by moving into it. Then the world is replaced, the user is transported into a manga, and their facial features are replaced with the large eyes, mouth, and cheek features typical in anime media.

How we built it

Development Platform We built the project in SparkAR.

The Portal The portal model was imported off sketchfab. Bingyu Li wrote javascript that keeps track of how close the user is to the portal. This is accomplished by measuring the distance between two null objects: one in the portal itself, and the other on the users head. When the distance is less than or equal to zero, the portal disappears and the scene effects become visible.

In the patch editor, this was accomplished by linking the script with a patch that measures less than or equal, and then creating visibility patches for each of the objects in the effect itself. When the value measured is less than zero, visibility is toggled "on".

The particle system utilizes a cherry blossom texture to make it appear as if cherry blossoms are emerging out of the portal.

Scene Effects

The face objects were designed in Maya, utilizing reference images from the internet. These were then painted in Substance Painter, while the object itself was modified using the SparkAR toolkit in Blender. These objects were then assigned a material in SparkAR, and textured using the Substance Painter export.

The effect employs a face tracker patch, which is linked to eyeball, nose, cheek, and chin patches. The respective objects are linked by position and rotation to the tracker patches.

Challenges we ran into

We initially wanted to employ plane tracking to place the portal wherever the user desires, but found that it made the effect confusing to use. Perhaps future iterations will allow the user a more intuitive version of this idea.

Another serious challenge was the difficulty of being unable to correct object origins in SparkAR. We had to employ the Blender SparkAR toolkit to correct this, and then export from Blender. The user really should be able to modify object origins in SparkAR.

Accomplishments that we're proud of

For the most part we were able to stay on schedule and work together to produce an interesting and fun SparkAR effect for Instagram Reels. We were happy to learn about the SparkAR development platform and develop a piece of entertainment technology. We were also able to utilize javascript in our effect, which greatly increased our ability to produce a highly functional filter. And it looks cool, too! : D

What we learned

The most important thing we learned was how to utilize SparkAR to develop AR filter effects for Instagram. This development platform is powerful and offers a highly functional environment capable of producing complicate AR effects.

What's next for Anime Portal

The next direction for this effect is to improve and expand its functionality. The filter should be able to represent a range of emotions, eye colors, and react to the user closing and opening their mouth and eyes. We would also like to implement custom placement of the portal, perhaps even utilizing multiple portals to provide a range of effects. While we did not have enough time to implement all these features in this iteration of the filter, we hope to be able to expand its functionality in the future.

Built With

Share this project: