In today's media-centric world, nothing speaks louder about a person than their online appearance: their profile picture. From social communities like Facebook and Instagram to professional circles like LinkedIn, users place great importance, pride, and judgement on the quality of their profile picture.
Though we all enjoy the impressive profile pictures of professional photographers and designers, not all of us are lucky enough to be so talented! Fortunately, propi.cc is here to help: by learning from selected sets of great profile pictures, propi.cc can help transform our ordinary, unedited photos into extraordinary online avatars.
What it does
propi.cc accepts user upload of a picture and finds the optimal way to crop it.
The user begins by uploading an image to through the elegant user interface at
propi.cc. From there, the user will be presented with the faces which are found in the image. By clicking on any of these, a face will be selected. The application will focus on that area of the picture and be cropped by the neural network for optimal appearance.
For a more technical description, read the following section.
How we built it
The project is hosted on Google Cloud Compute Engine, where the website was brought together, including a neat front-end. The back-end consists of two major components:
NEURAL NETWORK: We constructed a training dataset by collecting profile pictures from Facebook's Graph API. Using our own aesthetic judgements, we classified these as either "good" or "bad," with heavy emphasis on proper framing. Next, we trained a convolutional neural network constructed in TensorFlow in two stages: 1) profile pictures vs. background photos without people (for coarse training) and 2) good profile pictures vs poor ones (for fine tuning). In the end, this neural network is able to satisfactory generate ratings for profile picture candidates based on how "aesthetic" they are.
CLOUD VISION: Simultaneously, the back-end was constructed to make Google Cloud Vision API calls. These identified the presence of faces. The face bounds of each face were taken from the image annotations and cropped to present the user with and allowed for user interaction before passing to the neural network.
Challenges we ran into
Getting sufficient data for the neural network was a challenge: through Facebook's API, we were able to collect a little more than 200 profile pictures, which we had to manually label. This took quite some time. Configuring the neural network was intriguing and difficult as well, as it involved a significant amount of tweaking (layer sizes, stride size, pool size, etc.) in order to achieve effective results as well stay within resource limits (our computers kept crashing!!)
Accomplishments that we're proud of
Completing a finished project within 36 hours! A fully functional neural network was also very impressive. The neural network was built with an illustrative accompanying visualization, which could very well have been an interesting project in its own right. The project also made use of real-world data taken from Facebook and makes it feel very meaningful, with a substantial impact.
What we learned
An important lesson that we learned was that the quality of data defines the quality of the neural network and its results.
The APIs were fascinating and fun to learn about. In particular, they open up so many doors in terms of their versatility.
The Facebook graph, for instance, holds a wealth of interesting and real-life data which could provide exciting insights into connections which have never been explored before. From photos to videos to friendships, there is an abundance of data waiting to be analyzed on Facebook.
The array of Google Cloud products is similarly astounding. The machine learning APIs in particular provide an accessible and ready-to-use solution for novices to machine learning. The power that Google Cloud products provide, including but not limited to the ML APIs, pushes the limits of what hackers such as us are capable of.
The team has a broad range of experience levels and includes many first-time hackers. There was a lot to be learned by everyone.
What's next for propi.cc
The aspect of getting computer assistance in complicated design decisions opens many appealing avenues for application. For one, the algorithm that comprises propi.cc could very well become ubiquitous on social media sites as a way to give the user a default "suggested" crop/edit when they add a profile picture. This algorithm can be improved with more training data, including more fine-grained classification of different types and aesthetics in profile pictures.
But in the further future, similar machine learning algorithms may even help designers creatively invent new logos, layout publications, make art, document life through photography, and so much more. In this project, we've explored how even a mere 36-hour hack can reveal the incredible helpful potential of machine learning.