Most of us spend more time and are more intimate with our mobile devices than with other people - a fact. A lot of people are facing emotional problems, for example, anxiety, depression, and etc. - another fact. Why not collect data from our electronic devices, make sense of them, and make them helpful for us to be more aware of our recent mental status?

What it does

Mood Book keeps track of the data collected from online chat, keyboard and webcam - those that we usually do not notice yet have important role of indication of one's emotional status, - utilizes IBM-watson Natural Language Understanding to analyze them, and give you meaningful feedback on your emotional status.

How we built it

We used chart.js to visualize data and aos.js for scrolling effect in the front end. We wrote three different programs to retrieve data analysis result from chatting systems, captured facial images, and our keylogger log. The first two analysis results are from IBM Watson Natural Language Understanding and Microsoft Azure Face AI Emotion recognition.

Challenges we ran into

The API servers we used limit the amount of requests sent in a certain time interval, so we frequently get errors indicating too many request. Also we got several errors because the APIs restrict input format that's not explicitly documented. We managed to deal with the errors by filtering and tranforming the input data in advance.

Accomplishments that we're proud of

We are proud that we notice the connection between these under-noticed facts revealed from our electronic devices and made sense of them to help people be more aware of their emotional status, and furthurmore to take action should they notice an unhealthy trend. It is indeed a field that has great potential for us to explore.

What we learned

This is the first time we use APIs that require keys. It took us some time to figure out the difference between API and SDK, as well as how to apply for a service provided by a specific server and use keys to connect. Processing of data also requires accurate math model to make sense of the original data to the best.

What's next for Mood Book

we hope to develop a software that can automatically take pictures of users under their agreement and build a pipeline that can automatically analyze users’ facial expression and update the data to our website We’re thinking about building a random forest model to regress user’s self-graded scores on the data we collect, and analyze the casual effects behind. This can help user understand more about how things around them either influence or reflect their moods. Exert more efforts on the data privacy issues and protect users’ privacy.

Built With

Share this project: