This project was inspired by my idea that there should be a simple way for one to create a customized piece of art for themselves using AI techniques. I thought, what better way to combine unique text, images, and icons than focusing the project on Google Doodles?
What it does
- Downloads background image for subject and event.
- Applies the style of the event background image to the content of the subject background image ("mash").
- Saves an icon based on the subject.
- Classifies the icon as a G, o, or l, so it can replace that letter in the "Google" text.
- Selects a dominant colour from the icon.
- Creates the "Google" text with the dominant colour and calculate the position of the icon.
- Overlays the "Google" text on the mashed background and place the icon.
How I built it
- Neural Style Transfer to mash the subject and event image together.
- Google Images Download used to save the subject and event images to Drive.
- K-means Clustering to find the dominant colour in the icon, which is used in the text.
- CNN model trained by me with my own dataset to classify the icon as a G, o, or l.
- Flask with Ngrok to create web app where Doodle interface exists.
- Google Colab is the environment used to run the program with Flask.
Challenges I ran into
- Building a reliable dataset to classify the icons selected proved to be a challenge, as I had to set specific outlines for what a G, o, and l should look like to get reliable results.
- The iterations for Neural Style Transfer were not giving me the results desired, and were taking excessive amounts of time. This is why I had to use Google Colab enabled with the GPU accelerator and Eager Execution.
- Assembling all the components by building my web app through JS/HTML/CSS and enabling Flask with Ngrok gave multiple issues. For example, Flask with Ngrok often timed out and secure tunnels were rare. I had to learn how to properly utilize this resource with a Python program. I had to make the UI adaptable to any input image and screen size (progressive web app).
- Accommodating for different types (sizes, resolutions, etc.) of images was another difficulty. I had to ensure that the program found a suitable image before conducting the successive steps.
- Due to very recent updates in Google Images, the google_images_download library no longer supports downloading images. However, at the time the pictures and video demo provided were taken and the program functioned as outlined above.
Accomplishments that I'm proud of
- Making a unique final product that works based on any input image/query and has all the functions I imagined.
- Utilizes many Google platforms/resources (i.e. Tensorflow and Google Colab) to optimize speed and accessibility.
- Enforcing a reliable and relatively fast Neural Style Transfer technique.
What I learned
- How to train a CNN from scratch (creating the dataset, training it multiple times).
- Creating a progressive web app.
- Using Flask with Ngrok to deploy my program.
- Using K-means clustering to group images/classes together.
- Downloading Google Images dynamically.
What's next for AI Generated Google Doodles
- Adapting it to different forms of image download.
- Deploying the program to a Chrome Extension so it is available to more people through Google.
- Using a user's Google Calendar to figure out which subject and event to use to make the Doodle.