Inspiration
I was inspired to make this app to be able to help people who are visually impaired or have some form of disability to be able to interact more with the world. I realized that it is rather difficult to shop when you are visually impaired, and most individuals get assistance from store workers if they go to the grocery store. The idea is that they can use their phone's camera to capture images that are labeled according by training images with machine learning. Then with some form of headphones they can click on the screen to have the labels converted to speech.
What it does
Takes an image from your pictures or google drive to return the top three label results. These are taken from the Google cloud platform where they have pre-trained models. In addition, if you click on the button at the bottom it will convert the labels to speech with the confidence.
How I built it
I built it with Dart Programming with my favorite IDE Intellij. I used the flutterfire plugin for bringing firebase into the project. I implemented Google Cloud Platforms ML-kit from flutterfire which is the SDK for Google Vision AI for apps. I used used flutter_tts for the text to speech of the labels.
Challenges I ran into
There were a couple problems with getting flutter to work. Then one big problem is that there turns out to not be any Google Cloud Platform text to speech for flutter. It had an api for other programming languages. But I could not find. I spent lots of time trying to get that but I had to use the flutter_tts from dev.pub.
Accomplishments that I'm proud of
I just wanted to finish my first hackathon after I was a bit late starting and didn't really get to join any teams. I am proud of hacking my first app in the dart programming language with flutter framework. I am proud of getting the ML-kit to work with my project and for being able to implement the text to speech.
What I learned
I learned about google cloud platform, firebase, vision AI API, flutter framework, and dart.
What's next for banana
Integrating the camera into the app, so that you can take pictures and then it labels those images instantly and converts to speech the top three labels. I need to test it a little more in ios. I also want to add another feature for detecting text in images. Improving the labels to include more useful information about the images, especially if I continue going in the direction that it's main purpose would be to assist those who are visually impaired with grocery shopping.
Log in or sign up for Devpost to join the conversation.