Beautiful Seaweed Soup

Box plot
Scatter plot
Line plot
Product wordcloud Korea
Product wordcloud USA
Brand wordcloud Korea
Brand wordcloud USA

Inspiration

We were inspired to utilize the tools and techniques which we have learned related to software development, web design, and data analysis to implement practical solutions within the space of the online beauty product market.

What it does

For the implementation, we have used our original code to gather data on the 100 most popular beauty products on the Korean ecommerce platform, Olive Young. We have compiled data on product names, brand name, popularity ranking, access link, photo link, and prices, inclusive of current discounts.

How we built it

The program is built with Python, using the Beautiful Soup library. The code is tailored to access the data from the particular target websites used in our implementation. For further rounds of implementation, some basic modifications will enable the program to be applied more broadly to additional online platforms.

Challenges we ran into

The main challenges we ran into involved exploring and implementing new techniques for web scraping. The particular sites use JavaScript to load product info after the pages themselves load, so tools like BeautifulSoup do not work alone. This is because these tools only grab the raw HTML before any JS runs. We therefore had to learn Selenium during the hackathon.

Accomplishments that we're proud of

We are proud of our success in learning to implement new techniques while under the pressure of an impending deadline. We were compelled to solve the problem of websites which were not initially scrapable using the techniques with which we had prior familiarity.

What we learned

We learned new techniques for web scraping, including the automating of web scraping. We also learned how to deal with scraping website that use JavaScript. In addition, we explored new methods for generating data.

What's next for Beautiful Seaweed Soup

Going forward, we aim to implement a platform which enables users to review and compare top Korean beauty products across various online retailers. Our goal is to provide users with a means to review the products which are the most popular among specific product categories, while providing product-specific details and dynamic price comparisons across vendors.

We intend to connect buyers and sellers in a way which promotes the products on offer from sellers, while empowering buyers to locate their most preferred products at the best available prices.