Sign is more than an educational app, it’s a communication tool. Our development team was inspired by language apps that incorporate gamification like Duolingo, and educational language apps like Rosetta Stone. With a minimal user-interface, combined with a camera and video scan technology, Sign is great for those looking to learn sign language, and those looking to communicate using signing in everyday conversation.
What it does
Incorporating voice-to-text technology, and text-to-image processing, Sign translates spoken word to text and text to sign language, creating a fast and conversational way for beginners to learn sign language and an interactive way for more experienced signers to brush up on their skills. Sign also offers three tiers of language complexity so you can advance in your learning.
How we built it
Challenges we ran into
Accomplishments that we're proud of
Our development team had limited prior knowledge of mobile development prior to the hackathon. Given this, we’re proud of what we were able to accomplish in a short period of time, as well as the teamwork we exhibited throughout the process. We were able to develop a high fidelity prototype, and create app functionality like adding audio and listeners, adding camera functionality, translating voice to text using IBM API, creating mobile navigation, a splash screen, building a login system page, learning how to use gradient color on apps, and sending debugging request to service. We’re also proud of our idea which we believe if implemented, has the potential to make a positive impact.
What we learned
Our development team learned new technologies by using tools like react native, android emulator, expo, and IBM API. In particular, how Expo has made development easier and more intuitive (e.g. publishing app, app technologies such as audio and camera, multi-platform compatibility is automatic, React makes the app-building a little more intuitive). During Gear up we learned about different company APIs, and we learned how to incorporate IBM’s API into our project. We also made use of virtual project management tools and ideation tools like Miro and Lettucemeet. Our programmers learned about Figma and Design thinking and our designer learned about coding and programming, we feel this is valuable for better combining these two skill sets in future development work. We also learned a little bit of sign language!
What's next for Sign
What’s next for Sign? We weren’t able to fully develop all the functionality we envisioned. If Sign were to grow, we envision including increased levels of complexity beyond short sentences. We would also incorporate 3D technology and animation to better show hand movement, which is crucial in sign language. We would further develop gamification in the app to make the experience more engaging for learners. Future iterations would include user research upfront as well as usability testing. Our prototype also shows a scanning feature with the purpose of confirming a user’s comprehension. We would continue working on our UI to make the app more user intuitive. We would also implement the functionality we weren’t able to implement.
Prototype demo: https://www.youtube.com/watch?v=sKGbIfZu6UA&feature=youtu.be