An example of NutriText working.
We wanted to come up with a way for people who are keeping track of their nutritional information (dieting, diabetes, etc.) to easily get the information in a simplified way.
What it does
The user sends a text message with either a picture or text, and then the image (or text) is processed and the nutritional information is sent back to the user.
How I built it
We utilized the Twilio API along with python to easily send and receive text messages. In order to process images that we received, we utilized the Clarifai API to get a returned list of tags. We then sent this tag through the FatSecret API to get the list of nutritional information, and then this info is texted back to the user.
Challenges I ran into
We had some trouble getting the Twilio number up and running. After getting that working, we had issues correctly parsing the results from the Clarifai API in order to get the necessary tags. We also had difficulty finding a good nutritional info API, as we had intended to use CalorieKing, but this API required a paid subscription. We also had some issues handling both text and pictures.
Accomplishments that I'm proud of
Setting up a fully functional, programmable phone number and being able to identify an image, process it and then return useful data to the user.
What I learned
How to process an image, how to respond to text messages using Twilio, and how to connect python programs using the os and system libraries.
What's next for NutriText
We hope to find a more accurate image recognition API, as the Clarifai API is spotty at times.