What We Do
We'll pick recipes that you can make with the ingredients you own. FeedMe cycles through lots of recipes, supplying the full range of food you can make.
FeedMe uses machine learning and computer vision techniques to identify the ingredients you have. Simply take a photo of your groceries, and we'll update your in-app inventory.
Have you ever needed to borrow just that one extra ingredient? With Facebook integration, FeedMe makes it easier for friends share much needed ingredients.
How We Do It
Determining the groceries that you have from a single image requires a fair amount of machine learning. We initially take the image and segment it into separate clusters that each have approximately one food item. We then use a combination of a deep neural network and a series of support vector machines that generate the list of groceries found in your image. To get our training data for these algorithms, we just use the ingredient name and populate our training set using Google Images! During the classification step, we also use an OCR algorithm to extract any text we might find on the items' packaging and use that to easily determine the food item in question. To train the algorithms we used four g2.8xlarge (Graphics card based) AWS servers.
Our Own API
Although bringing home and organizing food may be a near universal occurrence, the amount of APIs for dealing with groceries is sadly lacking. In order to search for relevant recipes, we scraped data from SuperCook.com using PhantomJS and interfaced this with PHP to create our own RESTful API, hosted on AWS. This allowed us to query for things ranging from what ingredient we need to make marinated mushrooms (probably mushrooms), to how we can effectively share our groceries with our friends. Furthermore, we used AWS to support a LAMP and Node.js server to deal with queries to MySQL and our ML algorithm.
To top it all off, we integrated everything into the FeedMe iOS app. Using Swift, we pieced together a relatively complex User Experience that could be delivered to a wide range of consumers on Apple devices.