"SOAPBOX" World Effect

Not a soap, but a foam of expression. Craft a bespoke platform from your own photos/videos, then take the stand to defend it (free podium included).

Comment

(Judge for Emerging Track)

Inspiration

This was our team’s first time using facebook’s AR software, and we wanted to create an effect that would stand out. In the current social climate, the voices of the people are more important than ever. Our team came together to create an open platform available to anyone to raise their voice and spread their message to the world.

What it does

The filter places a podium in front of the user in the camera’s frame. It then projects an image/video from the user’s gallery onto a wall or other flat surface behind the user to help support their message.

How we built it

Our team utilized facebook’s Spark AR software, to create the Instagram world effect. The effect utilizes background segmentation to layer the user between the podium and the green screen object, which is selected from the user’s own photo/video library via Spark AR’s gallery texture. Plane tracking is used to project the image onto a wall or other flat surface. Further interactions were designed to allow the user to manipulate the podium’s position and size.

Challenges we ran into

Background segmentation ensures that the background of the recorded image, including the images that we added into the background, appeared behind the user. However, trying to place an object in front of the user while utilizing background segmentation to ensure other objects are behind the user at the same time was difficult. The solution we developed to overcome this problem introduced further difficulty with scaling and positioning the foreground object necessitating additional logic to accommodate. Furthermore, it was an ongoing challenge to scale and position the foreground object without affecting the background image at the same time. We would have liked to improve the plane tracking feature to allow the user to better manipulate the green screen object. Plane tracking is only possible with the backwards-facing camera, which limits the shot composition possible.

Accomplishments that we're proud of

We implemented both segmentation and plane tracking in a complementary fashion and overcame challenges associated with both toolsets. In addition, we created our own 3D models from scratch, using Rhino 6.

What we learned

As first-time hackers, we learned how to use Spark AR Studio and Rhino 6 to create our first-ever AR filter for Instagram Reels.

What's next for "SOAPBOX" World Effect

If we had more time, we would have liked to create a proper Instagram Reel recording. Since our effect was not approved for publishing at the time, we had difficulty saving the reel to include in our demo video. In addition, we would have liked to improve the plane tracking features to allow the user to better manipulate the green screen object. In the future, we would like to have the ability to add multiple images from our respective photo/video libraries as well as more objects, and the ability to manipulate these objects and images separately.

Built With

Share this project:

Updates