Inspiration/Goal

To minimize the spread of COVID-19 by reducing the number of trips people have to make to go to buy essentials when going grocery shopping.

What it does

  1. Reporting the availability of items in a store

i. After a customer has bought their items from a shop, they can search for their shop using the app.

ii. Then they can take a picture of the goods that they bought. The app will use ML to categorise some items.

iii. These will be added to a list of items (on top of a standard set of products such as hand soap), from which the user can swipe to set how much of that was in stock.

iv. This data is stored in firebase.

  1. Searching for items in nearby stores

i. In another scenario, a user can use the app to search for items that they want to buy.

ii. The app will search the database for a list of stores

iii. The app will ask for the user's permission for geographical location and use that to determine the stores nearby.

How we built it

We used React Native with Expo for the base of our app, along with its various packages and components. As for external services, we used a combination of Google Cloud ML Vision APIs, Google Maps APIs, and Firebase Realtime Database to store data. To test APIs we used Postman's robust set of testing features.

Challenges we ran into

Initially, we were planning on using a react-native package to run on-device machine learning. However, due to the heft and inaccuracy of the model, we decided to switch to cloud-based models. Initially, we had planned to "eject" our expo project into barebones react native, and after doing so we ran into many issues, costing us many hours of our time. However, we bounced back and were able to restore our progress in expo. This was our first time using React Native (although we have all done multiple projects in React.js)

Accomplishments that we're proud of

Deploying an app that not only uses Machine Learning models but also crowdsources the data necessary through the efficient use of databases. Clean and intuitive UI to set the pace for the users.

What we learned

React Native how to post and request from firebase to send and receive data. Also learned how to utilize ML models to sort vision data via OCR and Object Recognition.

What's next for ShopSafe

Integrate the google popular times data to show how busy each grocery store is. This will help inform people which grocery stores are currently high traffic, helping people plan their trip to minimize exposure to other people. Use OCR to get items from a customer's receipt.

Share this project:

Updates