What inspired me

I was inspired by the images I've seen other people create with VQGAN+CLIP popularized by Katherine Crowson's Google Collab notebook and wanted to see if I could automate the creation of new and interesting images.

How it works

It first imports text options from a google sheet and procedurally generates a new prompt to use for training. One of 4 pre-trained models (coco, imagenet, wikiart, sflckr) is selected and VQGAN+CLIP is then instructed to generate an image. On the 160th iteration, the output image is uploaded to social media. ex: AI Art Bot The python task (aiartbot.py) runs in an ECS container and uses EFS for persistent storage (output and models). The ECS cluster is comprised of a single dl1.24xlarge instance.

How I built it

I first created the Habana Labs Gaudi based Docker image locally for testing my code, then created the Launch template, ECS cluster, Autoscaling group and container task on AWS using spot requests to minimize my costs.

Challenges

I had difficulty getting Autoscaler to bring an ECS cluster from 0 to 1 instance when a Task is run. I ended up just manually changing desired instances from 0 to 1 to make an ECS instance available to run my Task as needed.

Accomplishments

This is the first project where I've used a google sheet to decouple text and config options from a program. I've had the idea for a while, but this is the first time I've used it and found it to be a novel solution for quickly making content changes without the need for code changes.

What I learned

Code for VQGAN+CLIP training is usually GPU optimized, so I had to make some changes to my Docker image and code to optimize for CPU training. GPU based training with VQGAN usually restricts the resolution of the output image based on the amount of GPU memory available, but I found that CPU based training seems to only be limited by available CPU/ RAM and can therefore generate much higher resolution output images.

What's next for AI Art Bot

AI Art Bot will continue running on a schedule and share its Art with the world on social media. You can find it on Twitter and Mastodon.

Share this project:

Updates