Today's world emphasizes equal opportunity for all, for people of different races, genders, economic levels, and much more, including ability. Despite that emphasis and efforts to make it a reality, not everyone in society is given the same opportunity, particularly for important things such as education.

We, the team behind Hello World, recognize these discrepancies, and strive to minimize these gaps between people all over the world.

Both Anton and Yair have family members that suffer from hearing disabilities, whose lives are harder because the world around them does not adapt appropriately for them. For example, communication is very slow for those family members because most other people are not fluent in any sign language, and written communication is simply slower than signing or speaking, which is not necessarily an option. The insufficiency in accounting for the hearing-impaired, further amplified by personal experience has inspired Hello World

🤔What it does🤔

Hello World translates sign language input (video) to audio or text, and vice versa. The output from audio/text to sign is animated, rather than simply sequencing individual pictures of hand signs.

🔨How we built it🔨

  • Built with JavaScript, TypeScript, HTML, SCSS
  • Libraries used: Node.js, Angular.js, JQuery, Tensorflow, Firebase, openpose,
  • APIs and Datasets: Google API: Compact Language Detector v3 (CLD3), WLASL (Pre-trained)
  • Other extensions: Mediapipe, Bergamot (Translation), SLP Pose Viewer, Human GAN, ServerSide LLM

💪Challenges we ran into💪

  • Slow processing/encoding (local), and high latency from website hosting
  • Slow Rendering due to unoptimized hardware
  • Buggy display (freezing, crashing)
  • Library dependency issues
  • Browser Support (Is only compatible with Chrome and some versions of Safari)

🏆Accomplishments that we are proud of🏆

We are proud that we were able to achieve our original goal with the application. Although the overall concepts involved in developing Hello World are not too complex, there are many components, and implementing so many of them and integrating them together in a time crunch was challenging, and extremely satisfying to complete.

Furthermore, we take pride in the potential of positive impact that this project has on the future. This achievement strengthens our belief that innovative technology can and should be a force for good, whether it is used for social or economic causes, or even things such as education and healthcare. We believe Hello World is one of many steps in the right direction, combining technology with empathy to contribute to a greater future.

🎓What we learned🎓

  • First, while working on the project, we researched a bit to learn more about deafness, and learned some statistics about hearing disabilities worldwide, which have influenced what we will add in the future.
  • Second, through the development process, we also learned (taught ourselves by trial and error) how to properly integrate some libraries and extensions used for language processing.
  • Lastly, as is the case with many team projects, the importance of cohesion, communication, and overall teamwork was very obvious. By constantly working together and complementing each other (through different areas of expertise), we were able to achieve a lot more than if we had worked individually and combined afterward.

🔜What's next for Hello World🔜

No matter how far we go, there is always room for improvement:

  • The processing for Hello World will be further optimized, so that it can take less resources and run more efficiently.
  • Multilingual support (different types of sign language as well as more written/spoken languages)
  • User customization (In conjunction with multilingual support, allowing users to choose preferred language for both input and output)
  • Automatic Input (Access to live input for more natural conversations)

Built With

Share this project: