My grandparents inspired my hack. My grandparents can't speak english and find it hard when visitIng the US, but they are comfortable using their mobile phones. I wanted to help them to make it easier for them to visit. This is how I decided to build an app which incorporates Google Cloud's Translate API to allow users to translate wherever they are. As I started building it, I realized I could help even more people by incorporating other languages and adding in fonts that make reading easier for those with dyslexia.

What it does

TextSlate has four main features optimized to create an easy user experience. First, is the text recognizer which uses machine learning to identify text in images and allows the user to copy the text to their clipboard and use it with other features in the app. The other features are translation, text to speech which helps non-native speakers better their pronunciation, and a font change designed to make text more dyslexia-friendly. TextSlate makes getting information easier for non-english speakers and those with dyslexia.

How I built it

TextSlate is built on Android Studio using Flutter and Dart. It is connected to Firebase which allowed me to use the Machine Learning kit provided by Firebase. It also uses Flutter's Text to Speech feature and* Google Cloud's Translate API*. These three components come together to provide the user with the easiest UI , pleasant layout, and helpful features.

Challenges I ran into

  • incorporating the Firebase ML vision
  • figuring out which development platform to use

A challenge I ran into was incorporating the Firebase ML vision. More specifically, formatting what was found in the image. I knew from the beginning what my goal with it was but didn't realize how hard it would be. I had to go through several round of debugging but my code eventually worked the way I wanted to. Another challenge was figuring out which development platform to use. I originally tried out xcode but found Swift hard to use. I did research on what other platforms could be used to create an app with the features I wanted. I soon found flutter and realized it was the ideal option to help the most people because it is compatible with multiple platforms.

Accomplishments that I'm proud of

My biggest accomplishment in this hackathon was getting an app up and running. I've never built an app before or even used flutter. I'm really proud that I was able to make an app which has all the abilities I had originally envisioned. I was really proud of how quickly I was able to get functionality working which included several cycles of research, debugging, UX design, app implementation, testing, etc. It was exciting to see that by simply doing what I loved, I could make such an impact in people lives.

What I learned

I learned a lot through my project for this hackathon. The first thing I learned was dart syntax. This was my first time writing code using flutter and dart. I also figured out how to use firebase and implement APIs. I've recently gotten very interested in the idea of using machine learning to improve user experience. This project is really special to me because it gave me a chance to actually get some hands-on use of machine learning. This hackathon gave me the opportunity to explore more APIs and expand my abilities past just desktop websites. At a personal level, I realised that when you're passionate and excited about what you do, hard work feels easy. I was up way past 2 a.m. last night and I didn't realize the time passing!

What's next for TextSlate

In the future, I plan to look into using a camera within the app itself. Currently, it needs you to take a picture before using the app, then upload it into the app to use the text recognizer feature. Adding in a camera would provide an even easier user experience by cutting down the number of steps necessary. While the app already uses speech tech, there is an opportunity to make it even more interactive by using this tech more extensively. I would also like to pay for full access to the Google Translate API and figure out how to implement the JSON file for it into the code. This will give users the option to translate any language into any language. This would widen the reach of TextSlate and help me help larger segments of the global population.

Share this project: