A conversation over Chick-Fil-A about a project involving image processing.
What it does
William & Mosaic can create a mosaic image out of any base image a user wants. Using a user-provided search term and base image, the project will scrape Bing for the search term and download images to use while tiling. After that, it matches the best scraped image to each pixel of the base image and replaces it, generating a mosaic image that resembles the original. Users can even upload their own images to be mosaic'd.
How we built it
William & Mosaic is built on a Python backend hosted on Heroku and uses CockroachDB to store small scale image information. We integrated several Python modules that we built with CockroachDB, Heroku, Firebase and a frontend webapp to store and lookup images, handle requests, and provide a user interface for people to submit images and search terms into. We started by writing the Python scripts and JS/Svelte frontend. Then, we set up the CockroachDB database to map RGB/LAB values to specific images. Finally, we integrated Heroku and Firebase for backend request management and image storage, respectively.
Challenges we ran into
Heroku provided some challenges while handling requests that had to be resolved. Our background tasks using celery ran off-machine, forcing us to look for off-board image storage. Thankfully, this could be remedied by using Firebase to provide a consistent way of communicating between worker threads.
Accomplishments that we're proud of
Tiling the mosaic by hand without using shading or overlaying the original image on top afterwards. The results we've been able to produce are incredibly close to the base images considering this fact. The frontend UI and webapp were constructed entirely from scratch and integrate seamlessly with the Heroku backend. Finally, our fuzzy matching algorithm to determine which image to substitute in per pixel in the mosaic allowed the project to proceed smoothly from the beginning.
What we learned
- How to use cloud-based database management and storage systems
- Tools and methods for image processing
- Algorithms for fuzzy matching
- Integration between backend and frontend including cloud services ## What's next for William & Mosaic We'd like to add more features to increase the accuracy and speed of the program. The matching algorithm could be revised to provide even better results given enough time. The tiling routine could be optimized so images could be delivered faster. Finally, we'd like to figure out a way to reduce bandwidth usage so we can make the most out of cloud-based services which limit bandwidth, like Firebase.