Web scraping can often be a daunting task, especially to the inexperienced. By providing a way to do this, especially with out inspector-like extension features.
What it does
Netscrape scrapes the net for content within certain selectors (similar to what you would use in CSS/SCSS or jQuery) to look up one or more fields on one or more web pages.
How I built it
We have a Chrome extension, along with a website running a Flask back end and React.js front end.
Challenges I ran into
Some websites generate different content for users, both logged-in and out. Furthermore, some of the
id names in the HTML are sometimes different based on the users, which means that our servers doing the parsing don't see the same content that our users saw.
Accomplishments that I'm proud of
Being able to automate the process of scraping websites, previously something that could only be done by someone with programming experience, or by paying thousands of dollars for one site.
What's next for Netscrape
Adding increased functionality and machine learning!