Inspiration
We believe that in the near future technology will have tremendous insight into our lives. We wanted to demonstrate a high-impact application of AI that has the potential to greatly impact people around the world with existing technology.
What it does
joyOS uses neural networks to predict your second-by-second emotional reactions based on the video feed from your computer's webcam. joyOS logs your emotions to help you understand your emotional state and what content/apps make you happier. It also tracks your smiles and prompts you to smile more!
How we built it
We used OpenCV to detect faces that are within a reasonable viewing distance of a computer webcam. Next, we used a neural network trained on tens of thousands of faces to predict facial expressions on a moment-by-moment basis. Finally, we display a video feed of our live emotion recognition in a web browser using the Flask web framework. We record the user's emotion information for future use.
Challenges we ran into
One of our teammates had to leave the hackathon early due to family reasons. We experienced great difficulty using Flask together with TensorFlow.
Accomplishments that we're proud of
We are happy we were able to get emotion detection working.
What we learned
We learned a lot about computer vision, neural networks, and happiness :)
What's next for joyOS
We are evaluating releasing a JavaScript app that can scalably run client-side emotion recognition.
Log in or sign up for Devpost to join the conversation.