knowron_LinguBridge

Inspiration

People all around the world have problems with language barriers. Most white collar workers speak English by now and can communicate through this common language. For many blue collar workers the communication is not as simple. We built an app to connect these workers regardless of their heritage and language skills.

What it does

There are two main screens:

A list and a schedule for the user/worker. Here the worker sees all of his appointment and can select one to report on.
After clicking on the appointment the worker can see a list of previous reports and his current task. He can then enter his report in one of 3 ways: a. Take a picture of his handwritten or printed report b. Speech to Text c. The standard of entering the text manually No matter the input, the worker always receives a transcript in the language of his choosing. Additionally he can view the old reports in his mother tongue at any given time. This way he can understand what has already been done and can change his work accordingly. To see how trustworthy the translation was, he can view 3 of our accuracy metrics. The final report for the manager can be read in English.

How we built it

As language models we used:

DeepL translates the text
spaCy: Natural Language Processing (NLP) which calculates the similarity between the original and translated text.
Chat GPT: also calculates the similarity between the original and the translated text
TF-IDF: to calculate the importance of a word in a given text
Cloud Translation API (google translate): a back translation to verify the similarity between the original and the translated text In the back end we used a flask server with a Planetscale database. In the front end we used React with Typescript with the library ant design.

Challenges we ran into

Since we had very little AI-experience, using the AI-models was a difficulty. With constant teamwork, dedication to the project and clean communication we managed to overcome this difficulty together!

Accomplishments that we're proud of

We are most proud of implementing AI into a project. Especially with our experience level, seeing a product work with this impressive technology made us all very proud. Additionally we are very happy about our development as a team. Before the Hackathon we were a group of friends with an idea, now we are a team who can work together amazingly. We will definitely work together in the future.

What we learned

We learned to much this Hackathon, it would be impossible to describe in a short list, but here are the main aspects we improved and learned:

implementing AI with API calls
creating a React front end
creating a back end with a server
team work

What's next for LinguBridge

We will continue to work together and implement more ground breaking technology. Being a part of such a fantastic team is something we do not take for granted. We will reach for the stars!