SignLink

Feeling unheard is an experience no one wants to feel. We wanted everyone to have a voice and be able to make an impact on the people around them without having communication being a major obstacle. When building our project, we learned that among deaf people, there are 70 million people who are considered culturally deaf. This means that they were deaf from birth or at an early age where they were not able to sufficiently learn the literacy skills of English. People who are culturally deaf primarily use ASL, or another form of sign language, to communicate, but a broad majority of the world does not know sign language. Although culturally deaf people are a small group, they still create the same effect on others when having to deal with a situation of emergency. If anyone is experiencing a medical emergency or a state of danger, culturally deaf people can still take action to save someone or themselves. Our project focuses on bridging this gap of ASL to spoken language by creating an ASL EMS translator that can translate speech to ASL and ASL to speech through a simple video call. We built the project with a huge reliance on AI. We used Gemini AI as our API for ASL-to-text conversion, and text-to-speech/ speech-to-text on the browser. Grok was used to make our UI as user friendly as possible. Both the patient's and EMS responder's UI include the camera of each other, common phrase buttons, and a primary communication(PC) button and display window. For the patient, the PC button initiates when the program should receive the sign-motion and their display window shows the ASL-translated words of the EMS responder. The EMS responder has the PC act as a mic and their display window shows what was said and if it has gone through. In addition, the EMS responder will be able to visually see and audibly hear what the patient has signed. Chert was used to make the FaceTime call more seamless and aid the additional cues (both are able to see what they have inputted into the system). Many of the challenges we face have come from our inexperience. With this being our first hackathon, many of the tools, AI's, credits, etc. were completely foreign in application despite our research on what were common tools. Despite this, we were able to get great guidance from ClaudeCode and use our background knowledge of this topic to create a product. The greatest challenge we faced wasn't our inexperience, but trying to figure out how to get the credits connected to the API's we needed. Although we were able to find help from the sponsor representatives and hosts of the hackathon, we still largely struggled and only successfully incorporated the AI's used, when our plan was to have more.