Cars.com Web Scraper

This tool provides valuable insights about the used car market. It helps visualize car listings to an individual. It is a great resource to aid targeted marketing and can help boost sales.

Comment

Inspiration

Visualization is key to attack the market. It helps focus our valuable resources in the right direction. I wanted to create a product that will help provide more clarity about the market.

What it does

It plots car listings on a map obtained from scraping cars.com

How I built it

I first started building the scraper with beautifulsoup. I also used geocoding to gather lat and long co-ordinates for each listing. Then, I accumulated the data in a CSV file. After that, I started to look for map plotting libraries and found leaflet. I extracted data from CSV using PapaParse and used it to plot the markers. To package everything neatly, I uploaded all the changes to git and implemented html templates for finishing touches.

Challenges we ran into

Js doesn't come with a CSV parser so I had to spend some time looking for Papaparse. One of the mentors Tristan suggested me to use it. I also tried hosting the project on digital ocean but couldn't do it.

Accomplishments that we're proud of

I am very proud about implementing the scraper using bs4. My scraper was so powerful that cars.com banned my IP :P which is why I had to use proxysite in my demo video.

What's next for Cars.com Web Scraper

Creating unique graphs and a nice dashboard. Adding more used car websites.

Share this project:

Updates