Visualization is key to attack the market. It helps focus our valuable resources in the right direction. I wanted to create a product that will help provide more clarity about the market.
What it does
It plots car listings on a map obtained from scraping cars.com
How I built it
I first started building the scraper with beautifulsoup. I also used geocoding to gather lat and long co-ordinates for each listing. Then, I accumulated the data in a CSV file. After that, I started to look for map plotting libraries and found leaflet. I extracted data from CSV using PapaParse and used it to plot the markers. To package everything neatly, I uploaded all the changes to git and implemented html templates for finishing touches.
Challenges we ran into
Js doesn't come with a CSV parser so I had to spend some time looking for Papaparse. One of the mentors Tristan suggested me to use it. I also tried hosting the project on digital ocean but couldn't do it.
Accomplishments that we're proud of
I am very proud about implementing the scraper using bs4. My scraper was so powerful that cars.com banned my IP :P which is why I had to use proxysite in my demo video.
What's next for Cars.com Web Scraper
Creating unique graphs and a nice dashboard. Adding more used car websites.
Log in or sign up for Devpost to join the conversation.