We wanted to do something related to data visualization and machine learning that would be interesting to scale. Understanding public opinion was an interest point, because its really difficult to determine popular opinion with the polarizing views published frequently on social media. We decided we would try to tackle this problem, and create a way to understand people's opinions on a large scale.
What it does
Our webapp uses a web scraper to pull recent tweets from a specified location based of a given keyword. With these tweets, we extract the useful information, and perform sentiment analysis to determine the emotional weight of the tweet. From there, the web frontend pulls the processed data, and displays it as heatmaps over the analyzed locations.
How We built it
front end - react map display - google apis tweet retrieval - python script tweet processing - node api sentiment analysis - IBM watson
Challenges I ran into
We decided to learn and work with unfamiliar technologies to strengthen ourselves as developers. Using react to create the front end was completely new for some of us. Using IBM Watson was something none of us had done prior to this event. We faced challenges in our lack of knowledge, and troubleshooting for long periods of time, but we overcame those to complete the project.
Accomplishments that I'm proud of
All of our web scraping and data processing is done by a separate node instance from the frontend, and is portable enough that it could be used as an independent API for a different project.
What I learned
We learned a lot about advanced front end frameworks like react. We also got experience with web scraping and data processing, which was very new. Lastly, none of us had used IBM watson. Overall, the entire software stack was a great experience to learn new technologies.
What's next for Hatemap
Friendlier UI/more input options and more accurate algorithm.