4th Wall AI

Inspiration

With technology we're so used to existing solutions that we sometimes forget what they could be. One aspect that modern devices and applications have failed to capture is the raw human connection. We were fascinating with the idea of creating lifelike AI agents to help us with tasks, and make much more personalized experiences. Software that actually knows you, and helps tailor the experience.

OpenAI's GPT3 model took the world by storm. We realized that it had the potential to create deeply defined personas through its story-telling abilities. The lack of deep connection between creators/founders/influencers inspired a search to leverage GPT3's powerful engine to solve this problem.

What it does

4th Wall uses the background and context of an individual to generate interactive Deep Fake models that can take a user input and hold a meaningful dialogue with a user.

How we built it

4th Wall systems leverage state of the art APIs to produce digital personas. We use GPT-3 to hold a conversation with users, Descript's overdub API to turn text conversations from GPT-3 into voice, and the Rosebud's tokkingheads API to create a deep fake video of our persona for the user. In the future, we can optimize this pipeline by using open source solutions for our voice and deep fake API so GPT-3 will be our only external dependency.

Challenges we ran into

GPT3 is great at holding conversation in a broad sense, but really struggles to represent a single person. We spent a large amount of time this week figuring out how to harness GPT3 to create personas representing a single person. Currently we're doing this by providing hyper specific background information of the individuals involved in the conversation, the context of the conversation, and the log of the existing conversation. GPT3 can then take this information and narrow it's focus, providing a consistent and surprisingly good experience for users. In the future, we hope to create a large dataset of conversations, and finetune the GPT3 model to become even better.

Accomplishments that we're proud of

Building digital humans. How freaking cool is that?! We had the opportunity to work with extremely cutting edge tech, and combine 3 different AI models into 1 consistent experience, all made within 4 days of development. Our team is awesome!

What we learned

The power of a team with diverse skill sets working together towards a common vision. We managed to get this built at a breakneck speed, and the energy unlike any other.