Facial Recognition Game

• 

  A capture from the game showing the tile layout

• 

  Code showing how I sent data from Pycharm to Unity

• 

  My camera showing how the computer sees the emotions in my face

Inspiration

I was inspired when I realized that facial recognition could be used on a Raspberry Pi. I ended up not using the Raspberry Pi, but I decided that I still wanted to make something using facial recognition.

What it does

It scans your face periodically and changes your character in game based on your emotions. NPCs will also talk differently based on what your current emotion is.

How we built it

I built it with python using FER libraries. I then used a local socket server to send that data to Unity. Unity then will see how the values are to tell what your emotion is. So, if your happy value is high, you will be happy and if your sad value is high, you will be sad. This then is used in Unity scripts to change the user's sprite and other features.

Challenges we ran into

I struggled with all the libraries in python when making the facial recognition since I didn't have much experience in using them before. I then had to learn how to use Unity since I had never used it before other than one small project in high school. My worst problem though was the project crashing and losing progress near the end of the time causing me to lose progress and the talking feature before presenting.

Accomplishments that we're proud of

I am proud of the final product being a functioning game with room to grow. I can definitely see myself improving on this idea and with more time, making a full Steam game with this concept with a full story using the AI face detection integration.

What we learned

I learned how to use python better and about how Unity objects work. I also learned about the design process and how you can pivot when you reach a problem. I had to change ways of solving problems lots of times since I kept hitting roadblocks. However, with enough time and dedication I was able to create a game that still works. In addition, working solo helped me see how useful being alone is for solving problems but also showed how hard it is to work on your own when you don't have any help.

What's next for Facial Recognition Game

I think with better art and more elements; the game can be better. I also wasn't able to make the dialogue as in depth as I wanted it to be due to time but with more interactions, your choices in how you show your emotions could lead to different outcomes and maybe take you down branching paths in the story.

Built With

• c+
• python
• unity