Inspiration
We want to create a search engine from a database scraping from walmart.com
What it does
The search engine will be optimized so that any keyword would return other most related keywords as suggestions
How I built it
We use selenium to scrape the data from walmart.com based on the category, then use TensorFlow to create an embedding from the text (product highlights of each item in this case), reduce the noise by sklearn PCA, then clustering the embedding texts.
Challenges I ran into
The team only composes of 2 people, so the workload is heavy for each, we didn't have enough time to scrape the data. We weren't able to scrape enough data (5000ish out of 20000). We didn't have prior experience in TensorFlow, that slows down the embedding process and reducing noise.
Accomplishments that I'm proud of
Being able to build our own crawler to scrape the website.
What I learned
Learn more about the search engine optimization process.
What's next for Walmart Challenge
We're still have to work on the embedding module, sklearn pca, clustering and generate GUI with a search tool
Log in or sign up for Devpost to join the conversation.