Instagram hashtags scrapper

Inspiration

The importance of data-driven decisions is becoming more important, and our team wanted to approach one of the more pressing issues. Through gathering and analyzing data, we believe, one can make better and more accurate business decisions. To do so, we chose to web-crawl one of the most lucrative and influential marketing tools, Instagram, and specifically its hashtag functionality to analyze data.

What it does

The user inputs ID, password, category word for search, and how many posts to scrape. Then, it automatically crawls all the hashtag data that are within the user-input category. For output, it gives an excel file with all the raw data and a key hashtag word that has the most usage.

How we built it

We used python for web scrawl as it is fast and has a lot of packages, such as selenium and bs4 that can help web scrape.

Challenges we ran into

We not only ran into various technical glitches but also could not finish our original project idea. We originally tried to add ML with skdlearn packages to analyze hashtags to determine whether it is spam or user-created. But due to lack of time and skills, we could not finish this.

Accomplishments that we're proud of

We knew that the best tool to analyze and gather data was Python, but we were both unfamiliar with Python. We are proud to learn and start from scratch to crate a functioning Instagram hashtag scrapper.

What we learned

We were both new to Python, and it was difficult to absorb all the new information related to data analysis. and not only did we learn the basics of Python but also Python's functionality as well as its web scraping tools.

What's next for Instagram hashtags scrapper

We want to continue what we could not finish in this short period of Hackathon: adding ML with skdlearn packages to analyze hashtags to determine whether it is spam or user-created. We would also like to extend our project into different social media to collect a broader scope of data.