Inspiration
Some of my friends and friends of friends have autism, and they were in mind when I was creating this project. For many people on the autism spectrum, recognizing and interpreting facial expressions is a challenge that often causes them to be misunderstood. Social cues that may seem intuitive to others—like a subtle smile or a furrowed brow—can be confusing or go unnoticed, leading to misunderstandings, feelings of isolation, or anxiety in social situations. These difficulties aren't a reflection of intelligence or empathy, but simply a different way of processing the world.
My app uses real-time facial recognition and emotion detection to act as a bridge between expression and understanding. By analyzing facial cues and translating them into clear information like emojis and word, the app helps users better grasp the emotional context of their surroundings. The app is simple to use and can be used in a variety of public or private situations such as at home or in a social gathering to make communication easier.
What it does
My app allows the user to detect 5 different emotions of a human face in real time--happiness, sadness, anger, surprise, and neutral, providing a simple and clear interface to facilitate smooth communication for individuals with autism.
How we built it
The app is made entirely in html, css, and javascript, using the ml5 facemesh library to obtain the coordinates of the various points on a face and using geometry to determine when the position of these points corresponds to the expression of a particular emotion.
Challenges we ran into
Determining which points/areas of the face to use and how to manipulate them to detect various emotions was definitely the main challenge in making this app.
Accomplishments that we're proud of
I have never worked with facial landmarking before, and I am proud to have gotten this emotion recognition working decently well by myself over the course of these 24 hours.
What we learned
Emotions and facial expressions are extremely complex, and we should be mindful of the difficulties some face interpreting these complicated social cues, and remember not to take our perception for granted.
What's next for MoodMate
Due to limited time, the app currently only detects 5 different emotions. I plan on expanding its scope to additional or more complex emotions such as disgust, fear, etc. as well as improving the app's accuracy in detecting these emotions especially at various distances and angles.
Log in or sign up for Devpost to join the conversation.