Bluprnt

Inspiration

The original inspiration for Bluprnt came from a TikTok that covered hairstyles for different face shapes. This seemed like a fun base for a project, as when cutting your hair, it's hard to really know what kind of hairstyle would look good on you without going to a professional stylist. It's also very difficult to determine your face shape from just the human eye. We thought this was an excellent opportunity to apply machine learning as it's great at picking up on subtle details like this.

What it does

Bluprnt is a React Native application that allows you to take a picture of your face, and receive your face shape, as well as hairstyles that traditionally fit well with that particular shape.

How we built it

Bluprnt is built using a stack consisting of React Native for the front-end, a machine learning model made using Keras and Tensorflow, and a Flask api written in Python to connect the two. The React Native application will access your smartphone's video, and allow you to take a picture of yourself. This picture is then encoded into base64 and passed to the Flask api. The Flask api has the trained model loaded onto it, which takes in the base64 string, converts it to bytes and then a numpy array, which is fed to the model to make a prediction on. The prediction consists of an array of confidence scores of each seven distinct face shapes. We take the highest confidence score, and pass it back to the React Native application, which displays your face shape, and several different popular hairstyles for that shape for both men and women.

Challenges we ran into

This was a pretty challenging project, working with apis in Javascript comes with its own complexities however whenever you add in machine learning on top of that, things can get crazy pretty quick. We actually made a well working machine learning model with a reasonably accurate score average, as well as a functional React Native application that can feed the image to the model- however at the end we had no way of hosting our api for connection. Due to the nature of the whole machine learning suite involving Tensorflow, Keras, and Numpy our api got pretty heavy pretty quickly. We ended up exceeding the size limit on our hosting service by ~70MB and couldn't cut down any further. Since we couldn't host our api, we tried local hosting both the server and front end, however because the React Native app is actually on a different device, localhost won't work and tamu wifi does not allow direct IP connections.

Accomplishments that we're proud of

We're proud of how far we got in using completely new frameworks. While both of us had a semi-grasp on how the technologies we were using worked, we had little to no hands on experience using the frameworks we used. Our machine learning model works with a good amount of accuracy, and the React Native application successfully transfers and presents its findings.

What we learned

There was a very steep learning curve for us on this project. For one, we finally have hands on experience with actually building a deep learning model using the Keras/Tensorflow suite. We also now have a good understanding of React Native, how it differs from React, and the different libraries available to do different things. We also dove pretty deep into api communication, and finally have a good grasp on where many errors come from and how to avoid them (which is really really awesome).

What's next for Bluprnt

With a paid storage service, we would be able to properly connect the app together. Beyond that, there's more we can do to the front end, perhaps displaying images of the haircuts listed. Perhaps even using a Snapchat filter like feature which would display the haircut right on your head to view.

Built With

• api
• deep-learning
• flask
• javascript
• keras
• machine-learning
• python
• react-native
• tensorflow