Explore Buzzfeed Topic

Inspiration

Art and deep learning together surely brings out the best of creativity.

Accomplishments that we're proud of

This is a topic that I once explored and tried but couldn't understand it properly and I hence read in depth about it today and it turns out to be pretty awesome!!

Prerequisites

Before learning this topic I already had an idea about Tensorflow, neural networks, convolutional neural networks , basics of machine learning so everything was not quite alien to me but understandable.

What we learned

ABOUT NST

Neural style transfer is an optimization technique used to take three images, a content image, a style reference image (mainly a famous artwork), and the input image that you wish to style up. They are blend such that the input image is transformed to look like the content image, but “painted” in the style of the style image.

To do so we’ll transform the base input image by minimizing the content and style distances (losses) with backpropagation, creating an image that matches the content of the content image and the style of the style image. (Quite technical topics like loss functions, backpropagation but lets not talk about them in order to keep it short)

IMPLEMENTATION

Pre-trained convolutional neural networks are used and this process is cut short by using concept of transfer learning where libraries like keras have provided us with these giants and let us experiment with them on our own problem statements.

alt text

Next, we will define the layers from which we will extract our content and style characteristics. We can make dictionary where we can map these layers and extract the outputs. Then we define the content loss and the style loss as shown below

alt text

Now we have computed both the loss functions. Therefore to calculate the final loss we will compute a weighted summation of both the computed content and style losses. Then the final integration of losses by traversing through the layers and computing the final loss by taking a weighted summation would give us our model. Finally, we would have to define an optimizer(Adam or SGD) that would optimize the loss of the network. And we are done!!

(This all is just a brief explanation there are many terms that I haven't explained here but I have gone through them separately)