Inspiration
I think I could've used something like this when I was in college. Any tool that would just help me understand one of my biggest, non-negotiable every day cost would be so helpful.
What it does
A person can enter a few ingredients for a meal (chicken, carrots, bread, etc.). The website will then crawl through Walmart's site, calculate an average price of all the products, and add each average price for each ingredient. It will return a total price and advise the user on whether it's cheaper to cook themselves or have something delievered using an app.
How we built it
I created a front-end using React. For the backend, I created an node.js server using express. It essentially acts like a web crawler behind the scenes, retrieving prices from Walmart's website, taking the average, and accumulating them into a total.
Challenges we ran into
I originally thought that the DoorDash API retrieved a store's prices. The DoorDash API is more suited towards buisnesses and stores trying to implement a delivery service, so I tried to think about a different way to approach that part of the implementation. Getting the web crawler to work correctly was most difficult. Having to constantly go back, analyze Walmart's html structure, and figure out a meaningful way to parse without relying on data that could dynamically change at anytime was difficult. I also had some troubles testing locally since you really need certificates and keys to test along the https protocol. I also faced difficulty finding an ideal place to deploy everything. I tried Google Cloud, but I was unable to navigate it well. While I wanted to try learning it, it was too difficult to navigate given time constraints.
Accomplishments that we're proud of
There's an implementation that actually crawls through the pages, aggregates the data, and sends a response. It's something live and interactive, and that's really cool! Anyone who knows about it could use it.
What we learned
- Never underestimate having to parse html structure
- While it can be cool to deploy on a major cloud provider (AWS, Google Cloud), sometimes it's more about accessing something that helps you build everything quickly and that's convienient to you.
- In general, working with the fast-paced web development technologies, there's going to be many deprecated classes/methods, code that throws unexpected errors, etc. Always being prepared to hit a brick wall is important.
- It's important to take breaks. There were times where I was definitely just spinning in circles, when if I just took a deep breath and did something else, the solution would've came quicker to me.
What's next for Cook or Book?
Many things are possible for this website:
- it can be expanded to look at various grocery stores (trader joes, krogers, etc.)
- possibly developing a database to store some of the data whether user or store could be great
- implementing more complicated web crawler logic where maybe multiple services are created to crawl through multiple pages
- doordash's website could probably be analzyed at the least to get some prices
- machine learning could be applied where the prices of an item are anaylzed and the user can see over time how prices might fluctuate
Built With
- axios
- express.js
- heroku
- javascript
- node.js
- puppeteer
- react
Log in or sign up for Devpost to join the conversation.