AI/ML-RADAR

Inspiration

Many of my close friends and my parents have suffered with various allergies ranging from food to general allergies in the air. Additionally, often we come across various foods that we may or may assume not to have any allergens in them however, unexpectedly, this assumption can result in a very serious sitatuation quickly. On a less extreme note, pollen and other allergies from pollution can result in irritation throughout the day, really disturbing many people. Due to the lack of applications which is able to detect the ingredients of foods and the allergens in the air, I felt obligated to create an application and my app development skills in order to create an app beneficial to those who have suffered and unfortunately resulted in extreme situations due to simple mistakes. As a result, I have created RADAR helping users detect food and air allergens based on preferences and locations.

What it does

The app consists of three different sections:

The main part of the app is the Food Allergen Detector.

This part of the app uses Firebases ML Vision Kit in order to classify the image of a food, with a food name in order to find the ingredient of the food which I will explain further in the how I built it section. You are prompted to select what your food allergen preferences are such as milk allergies, peanut allergies and more. After that, you are able to either input manually, by typing, a name of the food and see whether it has any of your allergy foods that you specificied in the beginning. You can also press the full ingredients button to view the full list of ingredients. For the image taker, you are able to press the take an image button, take a picture of your food and then the model classifies the food you took a picture of and then shows whether the food includes an allergen you dont wish to have.

The second part of the app is the air allergen detector:

In this part of the app, using your location, the app is able to find the local Air Quality index based on your location, In addition to that, it is able to show you a reference point such as good, bad or great in terms of air quality. There are also other options to visualize this data for pollen and pollution as well. You are able to press on a heat map and it will visually (Intuitively red being poor, and green being good) in terms of air quality for the air quality heat map and same in terms of pollen concentration for the pollen heat map.

The last part of this app is the procedures tab where you are able to view various procedures regarding how to tackle various situations such as detecting where you have an allergy or not, what to do when suffering or someone in front of you is suffering an allergic reaction. These various links allow the user quickly to access resource and act quickly.

How I built it

I built this app using Google's ML Vision Kit, Native Base UI Library, react-navigation, Breezometer Pollen, AQI and Heatmap APIS, and the Nutritionix API to reference the ingredients based on the name of the item.

For the first section I built the classification of the images and its results using Google's ML Vision Kit which allowed me to use an on board ML Model to classify the text of the food item which then later is searched with in the Nutritionix API in order to see if the food has the allergens I chose. I can also input the food name item manually, which uses Native Base UI elements.

For the second sections, the Pollen API, AQI Index and Heatmap's data was all created using Breezometer's Heatmap, Pollen and AQI Apis which I was able to retrieve the information from and then displayed. I also use expo permission to retrieve the location of the user.

For the last part of the app, it is a varied of links which I use Native Base library to create the ui for.

Challenges I ran into

I faced many challenges, in terms of dependencies and UI to be able to incorporate an on app ML-Vision Kit Model, which allowed the app to function so quickly. I spent quite a few hours resolving this and am happy I was able to get the project done. This was my first time dealing with Google Firebase's ML Vision Kit.

What I learned

I learned how to Implement Googles ML Vision kit inside of a react native application. I also learned how to pass data such as images and data between screens when navigating in between them.

What's next for RADAR

I hope to release this application into the App Store sometime in the future in order to help those around and allergic people themselves prevent, and raise awareness about allergies in general.

Built With

• apis
• breezometer
• firebase-ml-vision-kit
• ml
• ml-kit
• nutritionix
• react-native
• vision-kit