Location for Judging:
E14 building, 6th floor, Purple room, far back left corner next to windows
You forget someone's name. You don't know anyone at the party. You're not sure who's paying for the meal. Your ex is at the bar. Our inspiration comes from these every-day awkward situations, from all of those times when you say "Uh, I could've handled this so much better". Our goal is to give people practice and suggestions on how to handle these awkward situations, so that they can confidently say "I can handle this, I’ve done this a thousand times by now".
What it does
Our VR app simulates situations of different awkwardness levels. As users look around, we show them hints to get out of the situation, giving feedback, explanations, as well as things to stay to make the situation less awkward.
How we built it
We used Unity to develop on Oculus Rift. To understand human voice inputs, we used Azure Language Processing. To design and animate the characters, we used Blender and Cinema 4D. We also made real recordings to create the characters' dialogues.
Challenges we ran into
We had a hard time capturing Oculus Rift input, because we didn't use the Oculus Integration for Unity and we couldn't find good tutorials dealing with inputs. We also spent a lot of time discussing scenarios and deciding how we should design them to be both gamified and educational.
Accomplishments that we're proud of
Our application has some really cool scenarios that we're excited to show people! Also, our app can understand human voice input, catch where the users are looking, and uses real conversation records, creating a very natural interaction with users. We want to simulate realistic situations after all!
What we learned
- Creating realistic awkward scenarios in VR
- Uniting various story-telling, educational, and game design aspects in one project
- Creating a VR project from the ground up just in 3 days
- Writing and recording realistic conversations with real people's voices
- Capturing users' speech in VR using NLP
- Using gaze tracking to create more immersive, realistic, and responsive VR environments
- Balancing reality and gamified experiences when designing the visuals of environments and non-player characters
- Getting Oculus Rift inputs
- Working as a true team
What's next for Project Awkward
We see this project going a long way. We would like to create more scenarios to cover a wider range of situations, as well as use machine learning to produce natural continuous conversations between users and neural networks. It is also a possibility that the app will host multiple users in one scenario, possibly creating awkward situations among real users. Also, we would like to do more research to refine the interaction between users and non-player characters, so that the experience is as realistic as possible. The more realistic, the more effective!