The idea for Food Safe came from hearing our partner Zak talk about his dietary restrictions. Having nut allergies and needing to keep kosher for certain holidays Zak found it hard to determine which foods he could safely consume.
As a team comprising of Computer Science, Electrical and Computer Engineering majors we wanted to come up with an idea that allowed everyone to contribute to our design and learn new skills relevant to our respective disciplines of engineering.
What it does
The Food Safe bar code scanner is a tool that takes information given on food bar codes and lets the user know if it can be safely consumed based on a dietary profile created on an application.
The software allows the user to create their own profile on a web application. The profiles we used to demonstrate functionality include vegan, vegetarian, gluten-free, dairy-free, and nut-free. The user then can scan the food barcode with a computers webcam or a handheld barcode scanner. The end goal of the project would be to develop highly customized profiles. These profiles would include diets such as "macro-friendly", kosher, pescatarian, keto, low-sugar, and much more.
The hardware for the project contains four different colored LEDs along with an OLED Display. Green indicates a food is safe, while red would indicate that it is not. The blue LED indicates that the device is processing information. Lastly, the yellow LED is designed to indicate that a food is safe to eat but the user should use caution. In situates where the LED emits red or yellow. The OLED would display the reason it does not directly match a persons dietary profile or it does not fit the database.
How we built it
Our handheld keychain proof-of-concept was built using an Arduino Uno, a breadboard, a .96" OLED display and varying colors of LEDs. It utilizes serial communication to transmit and receive data between the C# code to move between different states on the LEDS and outputs on the OLED display.
The barcode detection software was created using XAML and C# that utilizes Image Processing Libraries to effectively detect when a barcode is displayed on a camera. This detection then turns the barcode into a value that directly corresponds to a food in the OpenFoodFacts database. From here we request the json related to the products information. We use a JSON Interpreting library in C# to read in tags and their values. Using some logic, we can determine whether certain ingredients would be a risk to the user and we can return that data back in a easy to understand format.
Our software for the app concept was coded using C# APIs, XAML, and AWS.
Challenges we ran into
For this hackathon our Hardware developer Zak had new tools at his disposal to work with, featuring an OLED display designed for Grove branded Seeduino units. Zak had troubles configuring this display to work with his Arduino Uno R3 and ended up manually wiring the screen directly to his Arduinos data ports instead of using the included cable intended for Seeduino units.
One of our goals for this project was to utilize AWS in a meaningful way. Our partner Andrew took the lead on this front on the cloud computing segment of the project. He was able to set up endpoints using AWS API Gateway. He was also able to return custom JSON data using AWS Lambda. He also leveraged AWS Cloudwatch to see the logs of the running jobs and diagnose problems. Additionally he was able to connect to a database hosted on AWS RDB to host user profiles and post new ones on his personal computer. He ran into problems when importing the requests and pymysql libraries though in Lambda. He used AWS EC2 resources to spin up an ubuntu instance to try to create his own lambda environment, but was restricted due to his permissions on the account.
After spending many attempts and being unable to make permissions to follow the advised tutorial, our team decided we had to give up the aws project since Lambda was essential to the entire cloud segment of the project. All the parts were working except for the import statements in lambda.
Accomplishments that we're proud of
Joey learned how to use XAML for this project and was able to create bar code scanning software with the use of a webcam Although Tommy had used XAML before, he had never made a XAML application from scratch before.
Zak was very proud of his capabilities with creating designs on the OLED display used with the handheld keychain design
Andrew was very happy with utilizing so many resources in AWS. Although the permissions to make the project working were restricted, all the components ran on the local computer to access the api endpoints and database.
What we learned
Zak learned how to properly connect an OLED display to the SCL and SDA ports on his Arduino device and use IC2 protocol to make custom displays
C# 6.0 has a great JSON Interpreting Library while older versions of C# have an older library which turned out to be much harder to use.
There were a few places in the AWS Free account where there were restricted permissions which made following certain tutorials hard.
What's next for Food Safe
If we were to continue with the concept of this project we would add additional customization to profiles and dietary specifications as well as producing an app that used its devices camera as a scanner and an all-in-one handheld keychain device.
We would like go in more depth with the science and accuracy of reporting on certain conditions. The database reports on nutrition facts, and we did not have enough time to use that data from the database. We were hoping to add options for other cultures and their dietary restrictions such as Halal options and other cultural diets.