iSearch

Inspiration

Have you come across an item that you want to buy but whose exact name you don't know? Are you sometimes not in the mood of typing in Google to search something in front of you? iSearch is an iOS app that will let you do categorize web searches in Home Depot, and potentially other websites using pictures that you take with your cell phone.

What it does

iSearch allows you search over Home Depot's website, using images that you take using your cellphone's camera. We outsource the image analysis to CloudSight and Microsoft Azure Computer Vision APIs to generate aggregated query terms and perform the search in the website of your choice.

How we built it

We developed this iOS app in Swift 4, using CloudSight and Azure Computer Vision APIs.

Challenges we ran into

Aggregation of tags/description of multiple computer vision APIs is a challenge itself, and we currently do so by combining the query terms that each API generates (up to a certain confidence).

Accomplishments that we are proud of

We made a prototype that works :D We integrated a couple of widely used object-detection tools in an iOS app that we think can be actually useful. We think a potential partner and beneficiary of this concept is Home Depot.

What we learned

We learned to code in Swift and how to use some cool computer vision APIs.

What's next for iSearch

Improve the query term aggregation with better heuristics for finding more relevant searches.
Integrate other potential websites of interest (e.g. Amazon, Yelp, eBay, Best Buy).
Provide dynamic recommendations about websites for which the given query might be more relevant (e.g. "pizza spaghetti Italian food" could suggest the user search in Yelp, All Recipes or the like). Some interesting ML model could help us do this classification.