'Rare pupper' is the academic term for an overachieving puppy. What better way to quantify the 'rareness' of a pupper than to learn an image embedding model that'll project any puppy picture into a condensed vector with fun geometric properties that allows for things like adding and substracting puppers in, what I like to call, rare pupper algebra.

What it does

Given any pupper picture, it will return a vector representation. This vector can be compared against other image's vectors using cosine distance to calculate similarity. Due to the geometric properties of our pupper embeddings, you can add/subtract puppy pictures to get new puppy pictures: it's rare pupper algebra!

(word embeddings but for puppy pictures)

How I built it

I built a pipeline to programmatically provision p2x EC2s (Tesla K80 GPU) using Docker-machine. I scripted a Dockerfile on top of PaddlePaddle's GPU Dockerfile, which I deployed to my p2x instances on top of Nvidia-Docker. I needed to configure OpenCV and patch a few BLAS incompatibilities but setting it up wasn't too painful.

I then needed a model. I didn't have the time for larger models, so I was looking to use the SqueezeNetwork. Unfortunately, no implementation exists for PaddlePaddle. So, I built my own, layer by layer.

I then trained it on the Stanford breed classification dataset. Though I only had one hour to fit and did not crop using boundary boxes (on purpose, because real-life puppy pictures won’t have bounding boxes), I got pretty good results: 12% mean accuracy across 120 categories with ~100 datapoints each.

I then spliced off the final layers of the models, stuck on new pooling and RELU layers, and trained the new layers on a contrastive loss.

What's next for RarePupperEmbedder

I will be exploring the military and industrial applications of this rare pupper technology.

Built With

  • paddlepaddle
Share this project: