Haloopinate

GIF
Haloopination of Mona Lisa (halfscaled to fit the 5MB file limit)
GIF
Haloopination of the TreasureHacks logo (halfscaled to fit the 5MB file limit)

Inspiration

Haloopinate was inspired by the recent and significant advances in deep learning for the visual arts, with models such as Midjourney. These models can contain billions of parameters, which would be too unwieldy to adapt for a one-day hackathon. In our brainstorming session, we recalled the older yet more tractable technique of DeepDreaming [1], which can produce wild psychedelic images that are both mesmerizing and disorienting. We thought that if we could continuously stitch together these images that it would make for a cool GIF, which ultimately led to the “inception” of this project!

What it does

Haloopinate allows users to leverage the power of deep learning to hallucinate an infinitely-looping psychedelic GIF based on an input image. It can be accessed through a user-friendly GUI or through a command-line interface, where more hyperparameter options are exposed.

How we built it

The DeepDream algorithm was implemented in PyTorch and Torchvision and is powered by a VGG16 classifier trained on ImageNet. We take inspiration from [2]. The website was built with React and Flask. The design was created in Figma and the styling was done in vanilla CSS.

Challenges we ran into

The DeepDreaming algorithm produces individual photos, so a major challenge was figuring out how to augment it to produce a coherent GIF with the desired effect (e.g., not straying too much from the source image). On the front-end side, we struggled with translating the Figma prototype of our website to a functioning React app. In terms of the back-end, the biggest challenge by far was connecting React to Flask and properly passing in the uploaded image to the deep dreaming algorithm.

Accomplishments that we're proud of

Alston: “This is my second hackathon and my first in a long while!”
Jamie: “This is my first hackathon, and I wasn’t sure about how to code our project on such a short timescale”
Joshua: "Learning Flask in a short amount of time and being able to put my growing frontend skills into practice to build the website are the accomplishments I'm most proud of!"

What we learned

How the DeepDream algorithm works
Connecting the website backend and frontend
Using Figma to design a webpage
Translating a Figma template to React
Parsing images and doing styling in React

What's next for Haloopinate

Haloopinate is very slow without a GPU, and future work could try to push its CPU performance. We could also tune and explore the impact of different hyperparameters in the DeepDreaming algorithm such as the choice of model backbone. We also plan to give users the option to specify a style (such as Starry Night) so that while the GIF is looping, it transitions into the specified style through style transfer.