There are plenty of Instagram scrapers on GitHub that will let you download a user's pictures or some metadata about their profile, but none are built with the capability of downloading the tens of millions of comments made on Kim Kardashian's over 4,500 posts. This valuable information is locked in a server farm, but could give us interesting insights about the way language has evolved since 2012, our relationship with celebrity, or how Kim K fans respond to different kinds of content.
What it does
Our project is two-pronged: the first part is a program currently running on AWS EC2 instances and has downloaded information about over 60,000 comments made on Kim's Instagram (and counting). Once we have all of this information downloaded in a couple weeks, we will be able to use D3 or other visualization tools to make (hopefully) cool observations. The second part is a VR installation that allows users to feel what it's like to be Kim K. Once a user puts on a headset, a library of 5,000 comments floats around them in an overwhelming cloud of speech bubbles. The VR experience takes inspiration from The Sims, The Secret of Monkey Island, and the iMessage app.
How we built it
Scraping data with Python and selenium running in a headless chrome browser, storing it in a MySQL database. VR experience made using Unity--mechanics programmed in C#, 3D models created in Blender, parsing Instagram comments provided in a CSV .
Challenges we ran into
Facebook servers throttling our too-frequent requests, deploying the scraping app on EC2 instances, trying to implement a text-to-speech feature in the VR experience.
Accomplishments that we're proud of
Doing this in a day.
What we learned
That we should make our Instagram accounts private.
What's next for Be Kim K: "be who you are, but be your best you"
Once we're done scraping all the data we're excited to dig into it!