Technology helped change the way we live our daily lives. From finding out information lightning fast to learning a new skill from the commodity of your home. Technology has made our lives easier and it's potential is yet to be unlocked. In times where everything is possible, we can help people with disabilities overcome their deficiencies.

What it does

ComKey Sign Language intends to help deaf people communicate with people with no auditive deficiencies. Moreover, our application aims to give a smoother transition, from natural language to sign language, to people that recently lost their hearing or speaking abilities. The core functionalities of the application are Speech to Text transformation and Sign Language recognition. With the help of his smartphone, the person can photograph/film the gestures of the sign language user and receive its textual translation.

How We built it

One of the core mechanisms of this application is the sign language recognition. The recognition is achieved through a image classifier written in Python. The dataset used for the training is provided by link. The mobile application was developed with the Android Studio IDE. The two applications exchange data through a firebase database.

Challenges We ran into

  • Tensor Size compatibility in convolutional neural networks (fancy words).
  • Changing the challenge after the first day.
  • Sleep deprivation. (A double-edged sword)
  • Exchange of data between applications.
  • Using Mac.

Accomplishments that We're proud of

  • Learning more about machine learning and trying to develop and adjust an image classifier.
  • Participating in yet another edition of HackaTUM.
  • Learning the sign alphabet.
  • Assembling Arduino hardware. (not related to this Challenge)

What We learned

  • We improved our teamwork skills and organisational skills.
  • We expanded our Computer Science knowledge (Machine Learning, Microsoft Azure, Android Studio).

What's next for Comkey

  • Hopefully we get to participate to a lot more Hackathon's in the future.
  • Maybe develop some more in the field of medical informatics.
Share this project: