-
-
Scraping Real-time Results
Inspiration
I have been working on Web Scraping from quite a long time and the LHD Build was the best platform for me to showcase my skill and make others learn, what I know!
What it does
The project contains code that scrapes Coursera Courses for given query and stores the result in a well-structured format. The code is scalable for adding more features time to time, and is developed solely for the purpose of learning.
How I built it
Technologies used are : Python, Selenium framework and Chrome driver ( for scraping), json and time module.
Challenges I ran into
The biggest challenge was to manage the website code that used to change for browser and driver. Therefore, I found a way to reach to the designated target data, using inheritance, instead of using XPATH.
Accomplishments that I'm proud of
I was able to complete the project even after those challenges and was able to record a video for teaching the basics of- How to scrape a website.
What I learned
I learned, how to develop a scalable code so that it can be used for a longer time without much updates, with extended functionality.
What's next for SCRAPER
I will be coming up with more such projects to deliver my knowledge and learn alongside. :)
Signing Off Apoorve Goyal
Log in or sign up for Devpost to join the conversation.