Autonomi

Inspiration

I was inspired to create this application after hearing about someone's struggles on losing their autonomy after losing their vision. I decided to search for a way using technology that might be able to help them.

What it does

Autonomi is a web and mobile application that uses machine learning to scan products and help the user purchase products without the use of a physical checkout or external assistance. The user simply scans a product to determine what it is, can choose to add that product to their cart and when leaving the store will checkout through the app receive an invoice sent from checkbook.io

How we built it

Autonomi is primarily built using React Native and Expo in JavaScript, I made the choice to use React Native because I thought it would give me the ability to iterate quickly and build for multiple platforms at the same time which is especially useful at a hackathon. In addition, the machine learning image labeling is done using the Google Cloud Vision API. The images are first uploaded to a Firebase storage bucket before the Cloud Vision API is called. In order to complete the checkout experience, I used the checkbook.io API in order to programmatically create invoice requests that are sent the the customer's email address.

Challenges we ran into

I had a lot of trouble with the initial setup of the android emulator. As I had primarily had experience in backend web development, however after lots of debugging I eventually got to the root of the issue. In addition, I wasn't sure where to store the images before sending them for processing at the Cloud Vision API, however thanks to the easy onboarding time on Firebase I was able to effectively deal with this issue.

Accomplishments that we're proud of

I'm proud that I was able to make a functioning cross-platform app in less than a weekend having had no prior experience in this field. I'm also happy that the Cloud Vision API seemed to be accurate on clear images which demonstrated that there was some viability to this project.

What we learned

I learned a lot through the course of this project, this was my first time working with both Firebase and Cloud Vision so I had some fun onboarding with those 2 technologies. I also learned about how useful android emulation can be on Windows, and how quickly cross-platform development can be started.

What's next for Autonomi

• Adding QR codes associated with different stores so that a customer can scan a QR code to get access to a store's prices and checkout.
• Image detection going beyond products, this could include people, documents, change, etc.

Built With

• checkbook.io
• cloud-vision
• firebase
• google-cloud
• javascript
• react
• react-native