SpeechForAllApp

Verbal communication is difficult for children that exhibit autism/speech disabilities. In their view, it's difficult to exhibit the full message to others. SpeechForAll will help eliminate this.

Comment

Inspiration

Inspiration for this project stems from my little brother, Ryan. Ryan has autism and at times exhibits difficulty in communicating with others. Over this summer, I spent a good amount of time with my brother, playing sports, and video games. However, I've come to realize the difficulty he has when trying to communicate with my parents, myself, and others around him. My brother isn't able to speak in full sentences and sometimes has trouble saying specific words. This can be troublesome when he tries to communicate with others that don't understand him as my parents and I do. This is when I came up with the idea for SpeechForAll. Pictures are a universal language, and through pictures, maybe my brother and those around the world that exhibit the same difficulty, and communicate more efficiently. This is my inspiration for SpeechForAll

What it does

SpeechForAll takes bits and pieces of implemented Google technology, to create an app that will help children with pictures. It allows children to have a gallery of photos that are custom to then, so they can communicate with others around them. For instance, if a child has difficulty saying "spiderman" and others around him can't understand him, the child can take out the SpeechForAll app, and show the person what he is talking about. Another feature lies with eh Google Cloud Vision API. If a child finds something in-person that they don't know and/or can't say, the child can take a photo of the object on the app, and the app will respond with the name of the object.

First, is Firebase for authentication. Users or parents will be greeted with a welcome page to either sign in or register for SpeechForAll account. For registrations, users should enter their first name, last name, child name, email, and password. After that is complete, the user is logged into their custom homepage. For the login screen, users will enter their email and password in order to have access to their custom home page.

After signing in, users will be displayed with their custom home screen. In the home screen, you will see images that relate to the child, whether it was previously saved or not. These images that are currently on the homepage are all retrieved from the Firebase Firestore. With the images on the screen, children can show other different images if they have difficulty pronouncing the item.

There are two buttons on the home page.

  • Take a picture button that allows you to connect with the camera on your phone and take pictures of realtime objects. The photo will respond with the name of the object in the photo. This was implemented using the Google Cloud Vision API

  • Next is the search button. This uses the Google Custom Search API, so users can search through images on the app. If a child wants an image of "Spiderman" on the app, the parent can search the photo on the app, and save it to the home screen.

How I built it

All functionalities were done on Android Studio using Java and documentation for Firebase

Challenges I ran into

There were many challenges faced with the full implementation of the application.

  • One is the additional ability to save images that the user takes a photo of. This can be useful for the child if he/she wants to use the picture later on.

  • Error handling of the login and register page. Unfortunately, I just ran out of time to fully implement the functionality of the login and register screen to make it as user friendly as possible. Errors are not caught and if the user enters bad information, the app may crash. Login and registration with perfect user input will work 100%.

Accomplishments that I'm proud of

  • I'm proud of the work done in this app given a short amount of time. Unfortunately, I had to work solo due to personal conflicts however, this was a project that I always wanted to get started on. After this hackathon, I'm excited to get back to work on this app and make the best it can be.

What I learned

  • I learned that coding takes patience. One can't just assume everything can be coded in a matter of minutes. Furthermore, I learned that one should always have a diagram or architecture of some sort. This definitely helps in boosting production.

What's next for SpeechApp

There are many more features to be added to SpeechForAll. This includes saving photos taken, making the UI more user friendly, and allowing a different method of registering. The app can allow users to register through Google, Facebook, etc which can make things easier for parents.

Built With

Share this project:

Updates