Inspiration
We've observed that if someone is not having the best day, a right dose of humor can make all the difference. It can not only brighten up a gloomy day but can also make a good day, great. So, we were motivated to create an end-to-end tool that used sentiment analysis to determine the user's mood and subsequently present them with aptly chosen media recommendations that can improve their day.
What it does
Our desktop application and Chrome extension use facial cues and the user's messaging patterns to conduct sentiment analysis. This data is used to show the user an xkcd comic that best suit their tastes at the moment. It can also be easily extended to other forms of media (like movies and books), provided enough time to scrape the data sources. But hey, who doesn't love a good xkcd! While originally made as an entertainment tool, the application actually has multiple advantages such as improving mental health.
How we built it
The Python application uses Tensorflow, Keras, OpenCV, and other dependent modules to detect the emotions on a person's face based on trained emotion models. Sentiment analysis was conducted using vader after the transcripts for all of them were scraped using urllib and Beautiful Soup. The modules fbchat and FBBack were used to obtain the chat transcripts of the user in their most recent messaging and another sentiment analysis was conducted for the same. We created our own-algorithm to match the sentiment data from the source (faces or messaging history) to the destination (the xkcd comic).
Challenges we ran into
None of the team members had prior experience in working with the modules they did. Figuring out how to scrape the data or conduct the sentiment analysis properly turned out to be a very contrived process, but the end result is something we are all very proud of.
What's next for ezxkcd?
We are aware we have created a product that can positively influence the mental health of countless netizens. So, we obviously want to expand both the domain and codomain of ezxkcd so that we can gauge more kinds of signs and signals to determine how the person of feeling and subsequently suggest more kinds of media for the person to enjoy.
Log in or sign up for Devpost to join the conversation.