We started with the idea of using the Microsoft Cognitive Services API to analyze news and social media text. We wanted to be able to be able to compare different sources across the Internet on the basis of content trending in their communities and attitude.

What it does

In a way, it helps support anthropology research. Our tool produces various visualizations about the popularity and sentiments surrounding trending topics on the web. Data can either be obtained for what's trending this instant, or analysis can be run on data saved from the past. The visuals allows for one to easily get a snapshot of online communities and interpret what is important to their members.

How we built it

We programmed in Python 2, which all members were fairly knowledgeable about prior to this hackathon. Retrieving text from Twitter and Reddit required writing web scrappers and accessing their apis. Natural language processing was provided by the Microsoft Cognitive Services API. Graphs were produced with the matplotlib and NumPy libraries. 1000 goldfish and 36 hours later, we finished our app.

Challenges we ran into

It took a bit of time of our team to get started-- we didn't come to the hackathon with any ideas beforehand. Once we settled on our project we still encountered issues with lack of prior experience, time (two members had other commitments during the event), and technical problems such as authentication, and data validation, which we all overcame.

Accomplishments that we're proud of

Seeing our app produce the first graphs was an exciting time-- even at that early time we were trying to interpret what it might mean about that corner of the Internet! Additionally, I think we all got to try something new, which was valuable.

What we learned

We learned about language processing and analyzing different kinds of data -- textual and numerical.

What's next for Twitter Reddit Trakker

Experimenting with more sources (such as news RSS feeds, Google Trends, Facebook, ect.), different visualizations, and finding notable posts and adding hover text to graph!!!

Built With

Share this project: