I was looking at myself in the mirror one day, and I thought to myself "Wow, if only I could beat up pizza toppings right now!" So then we did it, and here we are.
What it does
Our project provides a mirror that recognizes who you are, and allows the mirror to get to know you. It tracks your emotions every time you log in, and it even lets you talk to the mirror. The mirror then use a sentiment analysis to analyze your response to its question, and track your mood over time. In addition, we implemented two games, one which used facial recognition and emotion detection in order to put two players against eachother in a battle to mimic the right emotion. Then we used object detection to create a game wherein users could use their hands and the mirror to destroy pizza toppings without hitting the bombs.
How we built it
We used Python to make the whole application, and different libraries to accomplish different tasks. We used opencv to do object detection and tracking, and we used matplotlib to create a linear regression model which maps the points from the mirror onto the camera. In addition, we used a lot of Microsoft's cloud platform, Azure, in order to get our recognition algorithms. Using Azure we were able to obtain facial recognition data, sentiment analysis data, and voice to text.
Challenges we ran into
One of the largest problems we ran into was making the opencv module run at a reasonable framerate. With all the masks that we were creating in order to get the most accurate picture, we realized that our program was running at about 5 frames per second. It took us a while to optimize. We also had some challenges with project structure as we were learning more about what we wanted for this project as we worked on it.
Accomplishments that we're proud of
We were able to integrate with Azure, and we were finally able to create a working voice to text in order to get real voice commands throughout our project.
What we learned
We learned an abundance of things during this project including object detection in OpenCV, statistical modeling in python, integrating with Azure, and gained some insight about optimal project structure for future projects.
What's next for Pizza Mirror
We definitely want to add more games in the future,and a possible refactor for extensibility. In addition, we would like to seamlessly display the user's emotion data, so that they can see how they have been feeling throughout the week.