**See, Hear, Say Their Names**

As a tribute to Black Lives lost in US due to police brutality, my effect is to immerse users in remembering 93 victims in order to ease coping with unjust.

Comment

Inspiration

I decided to join this Hackathon to learn SparkAR and started with a “virtual presence” effect idea based on the gallery texture to offer an escape for people who got isolated due to Covid. Few days later, George Floyd got murdered, and my psyche changed. Instead of working on a project to technically challenge myself, I decided to build an effect for paying some sort of tribute to Black Lives lost due to police brutality.

What it does

See, Hear, Say Their Names immerses the user in acknowledging 93 African Americans killed in USA under police custody. The effect does the following: • User starts the effect surrounded by a “tunnel” of which walls are filled with the scrolling names ,and hearing two voices reading the names. • User can move inside the tunnel in three directions and rotate her camera up and down to read different names. • While "in the tunnel", user can take photos/videos of other people "outside the tunnel" through the tunnel doors. • While "outside the tunnel", user can take photos/videos of other people "in front of " tunnel doors.

How I built it

I used the SparkAR plane tracker in a different way. To enable user to move inside the tunnel in three directions, besides rotating her camera, I needed the plane tracker. However, I wanted the effect to start immediately, not waiting for a plane detection in the environment, so that the user finds herself in the tunnel when the effect begins. Experimenting with SparkAR, I observed the following: When a plane tracker is placed into the scene even without placing any object under the plane tracker, a horizontal plane at the device’s starting position is assumed, and SparkAR’s environmental understanding tracks this point.

I also wanted to offer the user some way of sharing her experience with this effect with other people by enabling her taking pictures/videos of people in the environment with the names on the tunnel walls. Since implementing this completely convincingly is still an open Augmented Reality problem, I implemented the following compromise:
1- When user is “outside the tunnel,” I used “SparkAR” background segmentation to take other people’s pictures as if they were standing in front of the tunnel doors covered with scrolling names. 2- When user is “in the tunnel," other people “cannot enter” the tunnel, but user can take their pictures/videos with the names as if they were outside the tunnel.

I implemented three different approaches for the first case, including a portal approach (see the Challenges section) to get a feel which one is more satisfying as an experience. The one I submitted is as follows: Once the user is outside the tunnel, she sees the tunnel doors as two screens, and can take pictures of others in front of these screens. I chose this option since it eliminates the possible experience that user may see others walking through the tunnel walls. (Of course, user still may see herself may walking through the tunnel walls// under the current state of AR).

User instructions: Not to interrupt the user’s experience especially inside the tunnel, I used only one instruction to remind about the audio. I also intentionally did not allow stopping he audio, but only lowering the volume by a screen tap, I believe AR experiences should be self explanatory and discovered by user.

To create the audio, I used text2speech.org free text to speech converter. I took the names from NPR.

Challenges I ran into

If I had chosen not to have doors, a good number of challenges I faced would have disappeared. However,, for the type of feeling I wanted to evoke, I envisioned a tunnel like structure with two openings at both ends.

To decide on what user should experiences when she is outside the tunnel,, I implemented the following approaches: First, I built a portal like implementation by occluding the tunnel and leaving only the doors visible from outside, so that when user is outside, inside of the tunnel is only visible through doors. Second alternative, I made the tunnel wall material with back culling, but door material as both sided. When user is outside and looking into the tunnel through the walls, she sees inside the tunnel like a portal as in the first case. But looking from a side, user can see inside the tunnel with that wall missing, which gives more background options for taking others' pics. Third and the submitted implementation is the one I described in the previous section.

Not a challenge, but time consuming task was to transform textures so that they exactly fit the structure and readable.

Accomplishments that I'm proud of

I am most satisfied making an effect for acknowledging 93 African Americans from Eric Garner to George Floyd, who died because of police brutality in US. Being "in the tunnel" surrounded with and hearing their names has not lost its power on me even after playing the effect many times. I believe it can also have a healing effect on other users.

What I learned

I learned SparkAR. I teach free of charge classes, and I always ask not only “what I learned”, but “what I can teach”? SparkAR can be a great tool to teach Augmented Reality.

What's next for Say

I’d like to make the name entry and audio to this effect dynamic, so that with unfortunate increase in the number of lives lost due to police brutality, and other unjust causes, this effect can be easily updated.

Built With

Share this project:

Updates