Inspiration

While there are many filters on Instagram, they are usually pre-defined. Even customers can adjust filters, they can only change colour, contrast and several simple options. Inspired by the great artists and their masterpieces in human history, along with the most recent development of deep learning and neural networks, we aim to create a webpage that help users to create filters with their beloved art style, or quickly apply multiple pre-trained art filters on their photos.

What it does

The project has a front-end, user-friendly webpage that allows users to upload an original image and choose multiple art styles out of the four available art styles. Or, they can upload an art image, such as Starry Night, or create their own new style. The backend programs contain a convoluted neural network and four pre-trained art models with parameters, adapted from an open source GitHub project. When the original image comes in, the neural network will change the image via backpropagation, until its metric of ‘Style’ is similar to the pre-trained models. If the user chooses to create a new art style, the neural network will train the style images in order to create the parameters of the model, which takes a much longer time.

How we built it

We used HTML5 to build the framework of ArtFilter and used CSS to style the webpage, successfully providing the users with a cherished opportunity to both view our existing model and design with their own models We utilized Python scripts, specifically Pytorch for the neural network. The python scripts are from an open source GitHub project about Neural Style Transfer: https://github.com/pytorch/examples/tree/master/fast_neural_style. The algorithm of Neural Style Transfer is based on “Perceptual Losses for Real-Time Style Transfer and Super-Resolution” (Justin Johnson, 2016). We utilized Compute Engine with 8 CPUs in Google Cloud Platform, in order to speed up the training process. We wrote a bash script to call different functions in the Python scripts so that we can apply multiple filters on one image.

Challenges we ran into

Training a new model with CPUs is relatively slow. Even with Google Cloud with 8 CPUs, it took around 8 hours to train 15.7% of the whole process and estimated 51 hours to train the whole model. We did not have enough time to integrate the user inputs from frontend with backend programs, but it is achievable if we have more time in the future.

Accomplishments that we're proud of

A user-friendly web page that helps to quickly apply ArtFilters and to create unique ArtFilter based on a given masterpiece Usage of Google Cloud App for demo webpage Usage of Google Cloud Platform- Compute Engine for speeding up training of new model.

What we learned

CSS3, Python, specifically Pytorch, HTML5, Google Cloud Platforms Compute Engines, Github Server

What's next for Art Filter

Speeding up our training process by utilizing GPU on Google Cloud - Compute Engine Write scripts to actually receive user’s input from website and server and integrate with the backend programs

Share this project:

Updates