We were inspired by the topics that we we have learned in class and we were also interested in web scraping bots. We learned about the tools offered by python in order to conduct the actual web scraping by importing selenium and pandas. In addition to learning web scraping with python, we also learned about developing a schema database for SQL, which was implemented in our web scraper, that allows our program to retrieve information that is requested in the instructions. The challenges we faced were establishing a connection between the webscraper and SQL database, establishing connection between our Java GUI and SQL database, and difficulty accessing information on websites that had files protected by HTML.

Share this project:

Updates