It’s a challenging time to take care of our mental health right now, with the COVID-19 pandemic. Many of us, fearful for our health and that of our loved ones, are confined to our homes. According to a study from Vermont Complex Systems Center, which they score the “happiness” of people's words from 50 million tweets per day, the happiness scores have hit it lowest this year.
Social media shouldn't be a front for us to show the curated parts of our lives. We should be allowed to be vulnerable on social media to receive the support we need. Our project picks out these key moments of vulnerability and provides support for the user to let them know that they are not alone.
What it does
With our project, we want to emotionally support Twitter users. We deep-dived into the tweets and performed sentiment analysis on it. To promote self-awareness and wellness, this is shown on a specially curated website for each user. Users get to relive the biggest moments in 2020 from their tweets on Twitter.
Utilizing our analysis, we categorized users into different emotional states and provide them with more support. For example, our chatbot will recommend motivational hashtags for stressed users to lift up their spirits.
How we built it
Built with ❤️ using Python, Flask, HTML, CSS, JS, NodeJS, PyTorch, and Firebase
Challenges we ran into
- Timezones - every member of our team was in different timezones and literally across the world. Logistics and meetings were a bit difficult
- Machine learning model was a huge file... so we used Git Large File Storage (LFS).
- Integrating the machine learning model to the Flask server and making sure the Twitter bot was calling the Flask server correctly involved a lot of debugging.
- Ensuring that the Firestore database was properly storing our data after parsing through all the Twitter API data
- Emotion Classifier: finding and building this classifier was one of the most challenging tasks the ML team had. Since we did not want to use standard sentiment classifiers that just classified text as positive, neutral and negative; we had to look for more advanced models that could classify a tweet based on the emotions in it.
The last challenge would be that as we were testing the bot, we would be restricted due to too many follows/unfollows on test accounts.
Accomplishments that we are proud of
- Submitting this project on time!
- Delivering and shipping a fully functional product
- Working remotely and internationally with team members of varying timezones! (GMT-8, GMT+1, GMT+8, GMT+5:30)
- Being able to debate and analyze key architectural decisions for each aspect of the project (discussions on how we decide to use a Firestore database with a Flask server)
- Being able to integrate sentiment analysis on the tweets with machine learning via PyTorch.
- Being able to successfully build the Flask endpoints that are utilized by the Twitter bot.
What we learned
- Collaboration with team members of different timezones and different cultures.
- Very educational discussions on approaches to implement our use-cases as well as designing our system.
- How important communication is especially for such a diverse international team.
- Getting Twitter 2.0 API takes a very long time.
- Learned more regarding the processes of the setting up of Flask servers, interacting with the Twitter APIs, deploying the servers, as well as associated frameworks/libraries utilized to support the machine learning.
- Machine learning takes A LOT of resources. We had to utilize Git Large File Storage (LFS) (https://git-lfs.github.com/) to version the huge files such as the machine learning model. We also could only train the model with 3k records.
- Learned that Twitter API doesn't allow us to create group DMs with a bot :(
What's next for #HowAreYouTweeting
- There are unlimited potentials for #HowAreYouTweeting. Our recommendation model can be used to pair up Twitter users for job opportunities, mentorships, conferences, and more.
- #HowAreYouTweeting is a yearly highlight of every Twitter user. Users get the opportunity to know themselves better and develop themselves holistically while using the platform. There could be different themes for every year.
- We also plan to optimize and personalize the recommendations more.
- If Twitter, in the future, releases the endpoint or API functionality for adding users to group DMs we would be happy to implement that rather than just sending a DM with a suggested user handle.
- Optimize Emotional Classifier and Scoring algorithm taking sarcasm and other factors into account
- Re-train model with the larger dataset (due to computational limitation, we could only train the model with 3k records)
- Find optimal values for the coefficients involved in the scoring algorithm