SignBridge

SignBridge

Inspiration

Our inspiration came from a K-drama called Twinkling Watermelon, which features a deaf girl as one of the main characters.

There was a scene that particularly stayed with us. After a boy was involved in an accident, she desperately tried to get help. However, when she picked up the phone to call an ambulance, she froze because she could not speak. The person on the other end of the call did not understand what she was trying to communicate.

That scene made us realise how difficult it can be for deaf or hard-of-hearing individuals to communicate, especially during emergencies when the people around them do not understand sign language. We felt that everyone deserves to be heard and understood, regardless of how they communicate.

This inspired us to create SignBridge — a platform designed to bridge the communication gap between the deaf and hearing communities.

What it does

SignBridge is a web application that enables two-way communication between sign-language users and people who do not understand sign language.

Using the user's camera, SignBridge can recognise sign-language gestures and convert them into text or speech in real time. This allows a deaf person to communicate their message to someone who may not understand sign language.

At the same time, the hearing user can reply back through text or speech, which SignBridge can present in a way that the deaf user can understand. This creates a two-way conversation instead of making the deaf person do all the work of communicating.

SignBridge also includes a basic sign-language learning feature, allowing users to learn common signs and phrases. This encourages more people to learn sign language and helps create a more inclusive community beyond simply relying on technology to translate.

Ultimately, our goal is not just to translate sign language, but to help people understand one another.

How we built it

We built SignBridge using Codex, with AI assisting us throughout the development process. As complete beginners to coding, we used Codex to help us understand, generate and refine different parts of the application. We went through several rounds of testing and refinement to improve the functionality and user experience.

We also developed different features to support our main idea, including sign-language recognition, two-way communication and a basic sign-language learning section.

Rather than simply generating the project and leaving it as it was, we tested the different features, identified issues and made adjustments to ensure that the components worked together properly.

Challenges we ran into

One of our biggest challenges was that we were a two-person team, which meant that we had fewer ideas and perspectives during brainstorming. At the beginning, we struggled to come up with an idea that we genuinely wanted to build.

Another major challenge was that both of us were complete beginners to coding and had zero experience participating in a hackathon. Looking around the event, we saw participants who were able to handwrite and understand their own code, including university students. Seeing this made us feel quite stressed and intimidated at first.

We also faced an unexpected challenge when one of our team members fell sick on the first day of the hackathon and could not attend physically. This meant that we had to adapt quickly and find another way to work together. Thankfully, we continued communicating through Google Meet and were still able to contribute to the project remotely.

Despite all these challenges, we learned to work with what we had, communicate effectively and make use of the resources available to us. The experience showed us that even without prior coding or hackathon experience, we could still turn an idea into a working project.

Accomplishments that we're proud of

We are most proud of the final result we managed to produce with SignBridge.

As Year 1 polytechnic students with little to no coding experience, seeing our idea turn into an actual working website was already a huge accomplishment for us.

We are especially proud that we were able to develop SignBridge beyond a simple sign-language translator. By incorporating two-way communication and a learning feature, we were able to make our original idea more meaningful and useful.

We are also proud that the hyperlinks and different functions of our website worked properly after several rounds of refinement.

Most importantly, despite one team member being unable to attend the hackathon physically due to illness, we were still able to communicate effectively and continue working together through Google Meet. We did not let the situation stop us from completing our project.

What we learned

The biggest lesson we learned is to take every opportunity to learn and step outside of our comfort zones.

Participating in this hackathon allowed us to challenge ourselves as Year 1 polytechnic students who are still adapting to a new learning environment. Initially, we were intimidated by other participants who had much more coding experience than us. However, we realised that everyone has to start somewhere.

We also gained basic coding knowledge through working with AI. Since we were required to explain the code and connect it to the features we built, we could not simply rely on AI without understanding what was happening. This pushed us to learn about the code behind SignBridge and understand how its different components worked together.

Beyond coding, we also learned the importance of teamwork and adaptability. Even when one of our team members was sick and could not attend physically, we found ways to communicate and continue working together online.

Most importantly, we learned that you do not need to be an expert before you start trying. Being complete beginners did not stop us from creating something meaningful. This experience taught us to be more confident in taking on unfamiliar challenges and to make the most of every opportunity to learn.

What's next for Sign Bridge

We hope to continue developing SignBridge beyond this hackathon.

Our next steps would be to improve the accuracy of the sign-language recognition system and expand the number of gestures, words and phrases it can recognise. We would also like to make the translation faster, more natural and more reliable in real-world conversations.

For the two-way communication feature, we hope to improve the experience so that conversations feel more natural and seamless, rather than simply translating one message at a time.

We also want to expand the sign-language learning feature, allowing users to learn more everyday signs and phrases through interactive lessons, examples and practice.

In the future, we hope to support different sign languages and potentially develop SignBridge into a mobile application so that it can be used more conveniently in everyday situations.

Ultimately, we hope SignBridge can become more than just a hackathon project. We want it to help bridge the communication gap, encourage more people to learn sign language, and create a world where communication is more inclusive.

Everyone deserves to be heard, and everyone should have the opportunity to understand.