Inspiration
After reading a blog post about automated music generation through machine learning, I really wanted to apply my knowledge of machine learning to develop an automated means of generating some form of artwork, like poetry or drawing. While I was searching online for direction on how to do so, I found out about that some people had used neural networks to transfer artistic style, which appeared to be extremely fascinating project to work on.
What it does
The web application takes in an image from the user and a stylistic choice as input. The user image is then saved to a local folder. The user image is then resized and reshaped. Finally, through a deep neural network, the stylistic choice of the user (e.g. Van Gogh's Starry Night) is tranferred onto the original user image. You can test your own images via https://styletransfer1.pythonanywhere.com/ !!
How I built it
I obtained the deep neural network used for the artistic style transfer through tensorflow_hub, a library containing open-source deep learning models. I then used numpy and tensorflow to resize the user image and the style image that would be input into the deep learning model. The user image, style image, and the transferred style image would then be saved locally into another folder. I used the Flask web-framework library in python to design the web application, which was built using HTML and styled using CSS, Bootstrap and Javascript.
Challenges I ran into
I had extreme difficulty integrating the flask web application with the deep learning model’s predictions. It was hard to find a way to send form data from the user to a format that the machine learning model can predict on. Another issue was that, for some unknown reason, after the machine learning model predicted on the first user image and style choice, it seemed to have cached the user image, the stylistic choice, and the transferred/merged image. So after the first input, the output was always the same. I solved this issue by saving the final model prediction into a single file, and appending a random integer to the end of the filename.
Accomplishments that I'm proud of
I am very proud that I was able to successfully synthesize the flask web application with the deep learning model. Before, I had only worked on small machine learning-related projects.
What I learned
I learned that in Flask, the file paths are very different from other web design applications.
What's next for Neural Artistic Style Transfer
In this project, I utilized the neural style transfer model provided by tensorflow_hub. Next, I plan on training my own machine learning model for neural style transfer. In addition, I plan on using React.js to provide a more user-friendly interface.
Log in or sign up for Devpost to join the conversation.