Screenshot of TRT in action! Figure shown is comparing various sports-related subreddits
Distribution of Tweets around 2am
Trending topics on /r/reddit-politics and their sentiments
We started with the idea of using the Microsoft Cognitive Services API to analyze news and social media text. We wanted to be able to be able to compare different sources across the Internet on the basis of content trending in their communities and attitude.
What it does
In a way, it helps support anthropology research. Our tool produces various visualizations about the popularity and sentiments surrounding trending topics on the web. Data can either be obtained for what's trending this instant, or analysis can be run on data saved from the past. The visuals allows for one to easily get a snapshot of online communities and interpret what is important to their members.
How we built it
We programmed in Python 2, which all members were fairly knowledgeable about prior to this hackathon. Retrieving text from Twitter and Reddit required writing web scrappers and accessing their apis. Natural language processing was provided by the Microsoft Cognitive Services API. Graphs were produced with the matplotlib and NumPy libraries. 1000 goldfish and 36 hours later, we finished our app.
Challenges we ran into
It took a bit of time of our team to get started-- we didn't come to the hackathon with any ideas beforehand. Once we settled on our project we still encountered issues with lack of prior experience, time (two members had other commitments during the event), and technical problems such as authentication, and data validation, which we all overcame.
Accomplishments that we're proud of
Seeing our app produce the first graphs was an exciting time-- even at that early time we were trying to interpret what it might mean about that corner of the Internet! Additionally, I think we all got to try something new, which was valuable.
What we learned
We learned about language processing and analyzing different kinds of data -- textual and numerical.
What's next for Twitter Reddit Trakker
Experimenting with more sources (such as news RSS feeds, Google Trends, Facebook, ect.), different visualizations, and finding notable posts and adding hover text to graph!!!