Bridge AI

An AI that gets rid of language barriers when viewing videos online. Allows access to content from across the globe and understand every word to full clarity

Comment

Inspiration

Living as international students, we were constantly inundated with vast network of culture and language, which led to momments where bridging the gap of context between cultures and language appeared to be essential. In the status quo, there was already a tiny window of opportunity to get to understanding different languages. However, we discovered that those translations on platforms such as youtube were just treading on a subpar standard, missing out on what the content has yet to offer. Globalisation has accelerated so many aspects of humanity and so the intercontinental connection between diverse individuals and cultures was also bolstered.The coming 4th industrial revolution, as many experts have postulated, will make the exchange of information and forming communities among people with such various ethnic background the most important code of life in the near future. The need for knowledge of culture and language has never been more clear than the current world we are living in.

What it does

We give accurate subtitles: by using a speech to text model we show the exact things the person in the video is saying.furthermore, we have an AI chatbot that can give a summary of the video (by looking at the subtitles generated by the speech to text model) and can discuss more about it with the viewer to increase its understanding of the video.There are things in certain countries that are culture based. like things foreigners aren’t quite aware of: idioms, phrases. The AI can be used to feed people with more necessary knowledge that would help them to understand the culture more easily. The page where you enter the URL looks like a search page, big input area in the middle and AI chatbot.

How we built it

We had divided this project into different tasks and completed them according assigned roles

#Renvitz

  • Used streamlit to design the landing page
  • Used streamlit to create chatbot interface
  • programmed the function of receiving the youtube url and displaying it on the following page

#Cosmic_perott

  • created mp3 download method for youtube videos
  • wrote the AI prompt
  • created speech to text model
  • created a method to connect the following model to AI

Challenges we ran into

For cosmic_perott, he struggled to get the transcript and mp3 file downloader to connect to streamlit. cosmic_perott also faced challenges when it came to installing certain libraries like yt_dlp and ffmpeg that is essential for mp3 download. There were a few moments where project almost broke down because streamlit always reruns the entire program instead of required areas. Also for ren_vitz he faced challenges when dealing with experimental_rerun as the pages had reload issues and compatibility with the version of streamlit he had. ren_vitz also had issues with using API key owned by a different person and connecting to his own device for the backend of the chatbot.

Accomplishments that we're proud of

we have built a project requiring a lot of different components in a short amount of time. cosmic_perott used a lot of different libraries that he is not familiar with and still created an end result. renvitz adapted to a different format to program in a short period of time. We both faced numerous errors during the process that was stressful but in the end we got over it and produced a solid product.

What we learned

On a broader scale we both learned how to efficiently manage a project and divide them in manageable blocks of task. We also learned how to efficiently cooperate as a team fairly taking part on given tasks. We also had to arrange every task for this hackafest online so it was a pivotal experience for both of us to communicate clearly with teammates.

On a more technical level, Renvitz had to adapt to a completely different format for coding and designing UI to what he was use to. Renvit also had to therefore, use resources on the internet and utilise them to make a product. Renvitz also had to research about the newly updated version of Gemini and how the whole google cloud operates to provide API keys and necessary settings. Cosmic_perott learned how to apply new technology he learned such as use of machine langauge and speech to text model to AI. It was a good chance for both teammates to apply what they recently learned into making a unique project.

What's next for Bridge AI

We are planning to upgrade the user interface by putting it as a popup icon so that it is more friendly and applicable for users in different websites not limiting itself to youtube contents but other news media. We will also install mp3 and make it function so that the translation is more accurate and facilitated. We will also call more advanced AI model such as openAI instead of gemini to improve the quality of response.

Built With

Share this project:

Updates