Driven by the desire to pair humans with their ideal adoptable pooch companions, we set out to enable people to search for and find dogs through a similar feature search of a photo of any dog, so people can find their very own adoptable fur-balls without having to know the breed or mix or to search through multiple adoption sites and photos of individual dogs.
What it does
Pawfect Match allows the user to upload an image of a dog, which uses machine learning to search for adoptable dogs who have a similar visual appearance through the Petfinder API.
How we built it
Using Keras with TensorFlow, we analyze images and data gathered from the Petfinder API and query a k-d tree to find the nearest-neighbors populated by mobilenet, a convolutional neural network that returns 1024-dimension feature vector for each dog image. The tree then returns dogs that have the highest similarity percentage to a user uploaded image of a dog. Utilizing Flask with a bootstrapped web application, Pawfect Match returns photos and links to the matched dogs’ petfinder pages.
Challenges we ran into
- Integrating the many libraries we needed caused dependency issues that required resolution
- The Petfinder API data was missing functionality that would have been useful, such as URL fetching by dog id
Accomplishments that we're proud of
- Getting all these technologies to seamlessly integrate with one another
- Using technology to help people find true happiness through new dog friends! :)
What we learned
We learned a ton about machine learning, and the challenges of working effectively as a team under pressure and no sleep
What's next for Pawfect
- Adding additional functionality to return more results and additional filters
- Building a specialized model for dog recognition (using attributes like breed)