As manufacturing technology progresses, so does the sheer number of chemicals applied to our body on a daily basis. Our cosmetic products contain all sorts of substances we can’t pronounce, let alone understand. With growing concerns about global microplastic pollution and new medical issues coming to light, it’s more important than ever to stay informed.

The customer doesn’t have the time to Google every single ingredient on their list. But with modern machine-learning models and web technologies, they shouldn’t have to.

That’s where Toxly comes in.

What it does

Toxly is a web app designed to put that information at your fingertips. It takes a photo of the ingredients list of a cosmetic product like a shampoo bottle. After some optical character recognition to digitize the ingredients list, it searches online for any concerns associated with each one. it produces an easily-readable list of issues with the product, ingredient-by-ingredient.

How we built it

There were 3 major parts to our project: the front-end, web-scraper, and OCR.

The front-end was designed to be scalable yet detailed. It was built with HTML, CSS, Bootstrap and ReactJS as a Single Page Application. It works as both a desktop and mobile site.

The Optical Character Recognition (OCR) was achieved through the use of Google Cloud Platform services. Google cloud vision’s text detection model was used in order to parse the ingredient list from the photos.

The web scraper was written in Node.js, as a Firebase web function. It searches through the web in order to find potential concerns with each chemical, and returns any relevant info to a web request.

Challenges we ran into

The time frame was quite limited, so it was important to keep our code modular and document everything. This allowed us to work on one piece at a time without constantly having to recompile the overall project.

Moreover, the website containing the data wasn’t exactly designed to be scraped. The information was all over the place, and some information was presented as an image which then had to be deciphered.

Integrating with Google Cloud Vision presented challenges in its own right. There were no packages for ReactJS to easily integrate with cloud vision, so we had to figure out a workaround by directly making HTTPS calls.

A surprisingly time-consuming part of this project was simply the plumbing: just trying to integrate all of the parts together to work seamlessly and create a streamlined experience.


Whether it be ReactJS, webscraping, or integration with Google Cloud Platform, all members of the group learned new technical skills. Perhaps the thing we learned best was the importance of collaboration and working together.

We are proud to have delivered a fully functional and fleshed-out product, especially given the short time-frame of the hackathon.

What's next for

We think that a product as useful as Toxly has a lot of potential in the future. We’re planning on continuing development of the app. Moreover, though the app currently scales well for mobile browsers, we’d also like to port it as a mobile app for the customers’ convenience.

Share this project: