William & Mosaic

W&M Cypher made out of images of campus
Bowser Jr. made out of images of Bowser
Bird made out of images of seeds
Our Heroku backend
Our Svelte frontend

Inspiration

A conversation over Chick-Fil-A about a project involving image processing.

What it does

William & Mosaic can create a mosaic image out of any base image a user wants. Using a user-provided search term and base image, the project will scrape Bing for the search term and download images to use while tiling. After that, it matches the best scraped image to each pixel of the base image and replaces it, generating a mosaic image that resembles the original. Users can even upload their own images to be mosaic'd.

How we built it

William & Mosaic is built on a Python backend hosted on Heroku and uses CockroachDB to store small scale image information. We integrated several Python modules that we built with CockroachDB, Heroku, Firebase and a frontend webapp to store and lookup images, handle requests, and provide a user interface for people to submit images and search terms into. We started by writing the Python scripts and JS/Svelte frontend. Then, we set up the CockroachDB database to map RGB/LAB values to specific images. Finally, we integrated Heroku and Firebase for backend request management and image storage, respectively.

Challenges we ran into

Heroku provided some challenges while handling requests that had to be resolved. Our background tasks using celery ran off-machine, forcing us to look for off-board image storage. Thankfully, this could be remedied by using Firebase to provide a consistent way of communicating between worker threads.

Accomplishments that we're proud of

Tiling the mosaic by hand without using shading or overlaying the original image on top afterwards. The results we've been able to produce are incredibly close to the base images considering this fact. The frontend UI and webapp were constructed entirely from scratch and integrate seamlessly with the Heroku backend. Finally, our fuzzy matching algorithm to determine which image to substitute in per pixel in the mosaic allowed the project to proceed smoothly from the beginning.

What we learned

How to use cloud-based database management and storage systems
Tools and methods for image processing
Algorithms for fuzzy matching
Integration between backend and frontend including cloud services ## What's next for William & Mosaic We'd like to add more features to increase the accuracy and speed of the program. The matching algorithm could be revised to provide even better results given enough time. The tiling routine could be optimized so images could be delivered faster. Finally, we'd like to figure out a way to reduce bandwidth usage so we can make the most out of cloud-based services which limit bandwidth, like Firebase.