Color Search

A Search Engine for images and colors

Comment

Inspiration

A friend of mine wanted to make a collage of their photo and a photo of the sky matching the color of their dress. That gave me the idea that maybe there's a website that finds photos like this. Even though there are websites that can be used to search based on color, there wasn't a single website that took a photo as an input. So, I decided to build one. The main algorithm used in the website is heavily influenced by this blog post

What it does

You can submit a photo as an input and then the website looks for photos (among the indexed photos from unsplash) and gives 100 photos that have the closest color palette. So, if you search using a photo that's mostly red, you'll probably get photos like the sky during the dusk etc

How I built it

I indexed the rgb photos of a certain number of photos and stored them in a sqlite database. I created the website using flask, where I take the photo and extract it's rgb values. Then I convert the rgb values to CIELAB values and then find their distance. The photos are then sorted based on distance to show the closest matches. The reason behind using the distance between CIELAB and not RGB values is that CIELAB better represent the human perception of color. Using RGB values would probably lead to a color appearing more often because they're closer in the RGB spectrum but wouldn't actually be closer in human perception

Challenges I ran into

Oh boy, A LOT of challenges. When I first built it, I was getting the color blue as a match more often than other colors. The reason behind this was when I was indexing the photos, I was clustering their colors to find the most dominant colors. In this process, the clustering isn't very accurate if the image quality isn't high enough. I still haven't figured out why this is the case but after I spent 2 entire days of frustration and troubleshooting, I figured it out and fixed it by increasing the image quality when indexing (should've realized the bug sooner).

Moreover, I had some trouble hosting the website. That's probably because of how new I am to web hosting.

Accomplishments that I'm proud of

I actually completed building an app that uses datascince in it!! I've never done anything like this before. I've tinkered around with datascience and machine learning libraries before but never made a product with it. I was really proud when I saw it work for the first time.

What I learned

I learned a bit more about web hosting. There's still a long way to go. This was the first complete project I built it flask. Previously I only had experience with django. So, it's nice that I can now use a light weight web framework too.

What's next for Color Search

I need to learn how to make this website more scalable. I did build a search engine. But it's more of a brute searching the whole database instead of using any efficient algorithm. When I indexed 10,000+ photos, it took around 40-45 secodns to return the results, which was way too long. So, I had to drastically reduce the number of indexed images to make it more usable. However I want to add way more images to it in the future. So, maybe I'll learn how to build better sql tables or learn how to use nosql databases (like mongoDB) to make this more efficient. I also want to add a secondary text search engine on the page that displays the matches to filter through the results using the images' metadata.

Built With

Share this project:

Updates