Performance Tunnel

Inspiration

A performing tunnel for aspiring performers at Instagram Reels.

What it does

With back camera, for a cameraman/woman to take videos of a performer inside a "Tunnel" augmented into the world space. The camerawoman can move inside the tunnel in three dimensions and rotate her camera in three directions. The walls of the tunnel store shots of the performer during his/her performance.

How I built it

I used the SparkAR segmentation, plane tracker and render passes.

I used plane tracker to enable camerawoman to move inside the tunnel in three dimensions and to rotate her camera in three directions. Intentionally, I did not place the “Tunnel” 3d-object under the plane tracker. This is because 1- plane tracking isn’t yet stable enough, and 2- when a plane tracker is placed into the scene even without placing any object under the plane tracker, a horizontal plane at the device’s starting position is assumed, and SparkAR’s environmental understanding tracks this point.

I used segmentation to enable the performer "inside" the tunnel to be visible to the camera.
I did not intend this effect to be used when the performer is “outside” the tunnel.

I used delay frames to keep snapshots of the performer on the walls. When there is no person inside the tunnel, the tunnel initially shows pics of the outside world. Once there is a person inside the tunnel and once the segmentation percentage is high enough, the performer is visible on the walls. If the segmentation percentage becomes low, the last snapshot with "ok" segmentation percentage is shown on the walls.

The back panel of the tunnels shows a concatenated snapshot of all the people inside the tunnel..

Challenges I ran into

Loading SparkAR effects to Instagram reels on Android phones. Instability of plane tracker. Glitches of segmentation.

Accomplishments that I'm proud of

Performing tunnel is a useful immersion for aspiring performers to use as a platform.

What I learned

SparkAR render passes.

What's next for Performance Tunnel

Using render passes and delay frames, walls of the tunnel can display more variety of snapshots of the performer and other people, e.g., including shots from the front camera.

Built With

• sparkar
• unity