Andrew Fung, Yu Hang Mak, Wen Feng Li, Wilson Liang SBU HACKATHON 2022
Challenge:
Provide users with a software that stores shoes into an inventory. The software records the model, ### size, and purchased cost of the shoe. It will fetch the market price of the shoe and compare the ### profit if the user would resel the shoe right now. Thus, makes life easy for fellow shoe resellers.
Next, is also the same goal but with pokemon cards
For shoes, we needed to scrape information off valuable sites like stockx.com, or ebay.com. Thus, ### we spent a bulk of our time with different languages: Java, python, javascript, and node.js to do so. ### Then we aimed to display this information on a website and decided on wix.com. While doing a ### web site, we came up with a domain name for it and used domain.com. While this was happening, ### the group members became very ambiguous and decided that doing the same with pokemon ### cards was the best innovative idea. For the Pokemon cards, we would only use Java swing (gui) ### and java to scrape pricecharting.com.
ISSUES AND SOLUTIONS:
We are 4 freshman with not much experience, and first time at a hackathon (It was fun, aside from ### all-nighters)
Stockx.com was extremely hard to scrape since we had to worry about it banning us and their ### methods of terminating bots. (It was easier with ebay/amazon). However, this task consumed
about 8 hours of our time somehow.
When we learned the introduction of node.js and its method of scraping ebay.com for shoe prices, ### it WORKED (displays in terminal)!! However, we created a html/css/js/node project in order to ### display this service. This is where it broke, after implementing the html files. We asked for help ### from 2 mentors
Theory 1: we must have a separate server where the server does the scrape and returns the ### ### information back to us.
Theory 2: Frontend backend not ideal in one file? Something about node.js conflicting with html?
We didn’t know that MLH gave off premium subscriptions to wix, so We couldn't bind our domain ### with our wix website. As a quick fix, We researched and had our domain do a redirect to the wix ### website instead. This introduced a flaw where stonybrook flagged this action.(Because our domain ### is http, redirecting to https).
Learning experience:
Node.js and how to scrape websites with it. Domains/ admin panel/ redirecting. wsl.exe.
Also messed around with: Velo by wix, cockroachdb, other languages to scrape websites
Log in or sign up for Devpost to join the conversation.