Both of us wanted to create something useful but also combine fields that we were interested in. Especially as Computer Science majors, we tend to spend a lot of time on our devices, constantly absorbing information from the web. We started to wonder whether that information has a visible impact on us. Of course, we assumed that our faces were drastically different when we were watching peaceful Minecraft videos to when we were
dying reading through a graph theory online textbook...but did any patterns emerge over time? Could identifying these patterns help us and other people tackle something as common as schoolwork stress to, possibly, even more serious situations that involve bullying and sensitive content?
What it does
RetraceSpace, or Retrace for short uses computer vision and natural language processing with Google Cloud to better your experience on the web. It does so in two parts: first, it tracks expressive reactions on your face as you use your computer, identifying joy, sorrow, sadness, or anger. When the content you're looking at evokes a particularly strong emotion, we run a sentiment analysis on the text and create a prevailing emotional opinion of what it is: perhaps you're looking at something overly negative like a hateful comment, or maybe you're reading a heartwarming story that's brought a smile to your face. Then we put the two together and compile it to give you an analysis that you can use to, at the very least, gain insight into your browsing habits, and at the very best, alter your habits to create a more positive experience on the web.
How we built it
Everything is available on our GitHub page: https://github.com/Gillgamesh/retrace-space
Challenges we ran into
There are definite privacy risks that we had to consider when starting this project as we are essentially constantly collecting user data in order to create the experience we are offering. In today's day and age, intrusive technology is a constantly growing concern. However, we decided to continue with our idea because we decided that there can be a lot of crucial benefits from it.
Accomplishments that we're proud of
We're proud of learning how to use Google Cloud - neither one of us had used the massive platform computing services before and it was definitely rewarding to see our idea come together with the services they were providing.
What's next for RetraceSpace
We want to run more analysis on the data we are collecting to give the user more feedback. At the moment we only had the time to give the user a simple comparison of their facial expression to the text that is on their page. We want to expand to other content like images, videos, and sounds, which are all things we chose to ignore. Then we can give users a more thorough breakdown of their browsing patterns, from being able to see their emotional log over the day to providing recommendations on their data. For example, if you, for whatever reason, you tend to watch a lot of puppy videos and have recently developed a pattern of negative reactions while watching them, we will suggest to avoid that content and give suggestions to watch more content where you show a habit of being happy to see.
We also want to create even more restrictive options for people who are sensitive to certain material. For example, we would place a temporary block on content that has time and time again caused a negative reaction in someone.