AI Witness News

The 'AI Witness News' hologram displayed in a Looking Glass frame.
Working with the data on a basic level
Certain trigger words will give us the search parameters for the ML models. Starting with sentiment then exploring others.

Inspiration

In response to the continued degradation of democracy as television news turns further into entertainment and social media algorithms favor sensationalism and clicks over educational and civic concerns we created an AI-enabled holographic avatar so that people can ask questions about how other people in a specific geographic region of the world feel about a current event.

What it does

A person approaches the hologram and asks it a question by gesturing with one hand over the indicated area to activate the hologram's microphone. The person then asks the question to the hologram. For example: "How do people in Harlem feel about Elon Musk?".

A voice-to-text translation occurs through Microsoft Azure Cloud Computing that then converts the voice into text. The text is then run through a Twitter sentiment analysis database indicating whether the sentiment in the area about the question is positive, neutral, or negative. This data is then sent to Azure and a call back is generated in a voice file available in multiple languages.

The hologram then answers the question with real-time lip sync.

How we built it

We started with a concept and then created the visual themes and personality of the hologram and then delved into the functionality of the substance of a person's interaction with the hologram.

To achieve the interaction we created a hologram design in Unity and display it on a Looking Glass Factory frame. After a person asks the hologram a question, a voice to text translation occurs through Microsoft Azure cloud computing.

Once the voice is translated to text it is sent to a sentiment analysis database that analyzes whether a sentiment in a geotagged region is positive, neutral, or negative.

Once this information is known it is translated back into text and an Azure call back is generated in a voice file available in multiple languages.

The hologram then responds to the person

Challenges we ran into

Completing all the specific concrete processes for the vision. We created the basic functionality and then had to go back through to ascertain what was possible and what was not in the pursuit of the final vision. We are fairly confident with what we have created so far that the full vision can be made.

In terms of more specific and practical things, Twitter required a more advanced developer access than we were able to gain access to for the prototype to fully function.

Accomplishments that we're proud of

We worked with Leap Motion, Microsoft Azure, Unity, and Looking Glass Mirror and integrated these systems in a coherent and functional way.

What we learned

That with a little more expertise and API access our hologram can work in the world!

We also learned how to start with a huge idea and then how to create small steps to create a highly complex concepts using new equipment and multiple APS.

What's next for AI Witness News

We plan on presenting the hologram during live events at cultural institutions in Los Angeles as well as starting to experiment with making it a streaming hologram that can interact with people in real-time.