We love futuristic, sci-fi, scary movies and games, so what better than to make a VR game that tests the limits of your bravery? We also wanted to explore the way in which neural networks can be utilized to make games personalized and adaptable.
What it does
As the player, you goal is to make it out of the hall alive, and in order to do so, you must remain as calm as possible. We’ve trained the system’s AI to tell the game when to send out more thrills and jump scares whenever you start feeling more relaxed, so do not think this will be an easy endeavor.
To reach the end of the game, the player must collect all of the tokens in the hallway to unlock the door to the next level. On their way, they’ll be confronted by scary monsters, and their own fear. The only way to exit the final door and get free is to remain calm.
How we built it
We used a Muse headband to collect frequencies on five major brain waves: alpha, beta. gamma. delta, theta. The Muse sends data in a stream of OSC (open sound control) messages with with we hosted on a python server. On the server, we parsed the data and organized it into a matrix-like data structure to train out ANN (artificial neural network). We trained the network by wearing the Muse headband and watching a calm, funny, and scary video. The output values of the ANN were then sent to a server on the Unity platform where they could help determine the “scare factor” of the game in realtime.
The frontend of FEARME is in C#, and was built in Unity. Though none of us had any VR experience, we were able to develop a game that is personalized based on your how scared you are (using values we received from the backend). We displayed how scared you are, and modified walking speed, the player’s breathing sounds, the size of monsters and the speed of the zombie based on your fear level and the normalized data from the neural network.
Challenges we ran into
The hardest part of the project was learning how to use Unity. None of us had prior experience, so figuring out stuff like better controls, porting to the Oculus Rift, making collisions work, was quite difficult. We also faced issue with the collision boxes of the walls, since the player would be able to leave the bounds of the level, but not return back into it. Another issue we had involved getting the scene transitions to work.
We also faced a few issues with merge conflicts, but we were able to develop a smooth workflow to avoid conflicts towards the end.
Accomplishments that we're proud of
We actually coded and trained an artificial neural network! and for the first time! Not only that, we were able to create a game that was actually pretty scary. Our focus group can definitely attest to that :/
What we learned
All of us went into this hackathon knowing nothing about Unity, VR, and very little about creating and training an Artificial Neural Network. So, we are all super proud to say we learned a LOT about game design and ANNs.
What's next for FEARME
We want to enhance the game experience by adding more features the extend the range of scariness. We want to increase the personalization of the game, to make each experience each time you play it more unique. We want to implement alternate game paths depending on how scared you are. We also want to improve the ambience by making the flashlight flicker, adding more sound effects and challenges.
We also want to create a frontend for our game, that shows you statistics on how scared you were throughout the game (highest/lowest scare level, median scare level, etc).
On the backend, we hope to add precision to the neural network by obtaining data from other emotions such as anxiety, excitement, etc.