• 

  Main Launch Page

• 

  AI translated transcript page

Inspiration

Living as an international student, I was constantly inundated with the vast network of culture and language, which led to moments where 'connecting' the gap of context between cultures and language appeared to be essential. In the status quo, there was already a tiny window of opportunity to get to understanding different languages. However, I discovered that those translations on platforms such as youtube were just treading on a subpar standard, missing out on what the content has yet to offer. Globalisation has accelerated so many aspects of humanity and so the intercontinental connection between diverse individuals and cultures was also bolstered. The coming 4th industrial revolution, as many experts have postulated, will make the exchange of information and forming communities among people with such various ethnic background the most important code of life in the near future. The need for knowledge of culture and language has never been more clear than the current world we are living in.

What it does

Connect AI gives accurate subtitles: by using a speech to text model it shows the exact things the person in the video is saying. Furthermore, ConnectAI can be used as a chatbot that can give a summary of the video (by looking at the subtitles generated by the speech to text model) and can discuss more about it with the viewer to increase its understanding of the video. There are things in certain countries that are culture based. like things foreigners aren’t quite aware of: idioms, phrases. The AI can be used to feed people with more necessary knowledge that would help them to understand the culture more easily. The page where you enter the URL looks like a search page, big input area in the middle and AI chatbot.

How We Built It

I used streamlit to create a platform where the AI's results are shown and where the user can communicate with the AI (including receiving URL from user for video and displaying it on screen) Then, I created a speech to text model using the faster-whisper library in python Next, I created a mp3 downloading function to extract a pure mp3 file that the speech-to-text model will use to extract text. the text will then be used by the AI to be translated into the desired language

Challenges We Ran Into

I struggled to get the transcript and mp3 file downloader to connect to streamlit. I also faced challenges when it came to installing certain libraries like yt_dlp and ffmpeg that were essential for mp3 downloads. There were a few moments where project almost broke down because streamlit always reruns the entire program instead of required areas. Furthermore, I faced challenges when dealing with experimental_rerun as the pages had reload issues and compatibility with the version of streamlit I had. I also had issues with using an API key and connecting to my device for the backend of the chatbot.

Accomplishments We Are Proud of

I built a project requiring a lot of different components in a short amount of time. I used a lot of different libraries that I was not familiar with and still created an end result. I adapted to a different format to program in a short period of time. I faced numerous errors during the process that was stressful but in the end I got over it and produced a solid product.

What We Learned

On a broader scale I learned how to efficiently manage a project and divide them in manageable blocks of task. I also learned how to efficiently cooperate on given tasks.

On a more technical level, I had to adapt to a completely different format for coding and designing UI to what I was use to. I also had to therefore, use resources on the internet and utilise them to make a product. I also had to research about the newly updated version of Gemini and how the whole google cloud operates to provide API keys and necessary settings. I learned how to apply new technology I learned such as use of machine langauge and speech to text model to AI. It was a good chance for me to apply what I recently learned into making a unique project.

What's next for Connect AI

I am planning to upgrade the user interface by putting it as a popup icon so that it is more friendly and applicable for users in different websites not limiting itself to youtube contents but other news media. I will also install mp3 and make it function so that the translation is more accurate and facilitated.

Built With

• css
• faster-whisper
• gemini
• html
• python
• streamlit