Main page with tables
Extract postal code from the address and get the average price
Original scraped data
Through this Hackathon, I would like to enhance my skills about Data Science. The project is therefore related with the topic of data science.
What it does
The python code will parse all rent information in the area of greater Montreal posted on Kijiji, and get an average price of rent. On the website, it will visually show but could not be completed.
How I built it
For web scraping and data parsing, I used Python and library called Beautifulsoup and Pandas. Basically, the result of web scraping is stored in a CSV file. For front-end, I used Angular with Angular material for table. I used d3.js for getting data from the CSV file.
Challenges I ran into
Website structure was hard to scrape; therefore, to get each attribute was challenging. I try to have a different kind of tables but for this iteration, I end up with average price for rent.
Accomplishments that I'm proud of
I participated in many Hackathons and this is my first time to complete a project alone. And I have learned a lot of knowledge about web scraping and CSV format.
What I learned
I have learned a lot for web scraping and CSV formats. Getting an experience of using Pandas and Beautifulsoup is also a great opportunity that I have.
What's next for Data Parse from Kijiji
Getting specific google maps (or links) in the column of the map and visually showing where it is located. Currently, to know the neighbourhood, I need to pay to Canada Post. Basically, I could not complete all the functionalities. Next iterations will be more including the jobs in the front-end part. Could not efficiently get Google maps as well.