Dub The Valley

Inspiration

We were sitting in one of the rooms in the IC building of UofT Scarborough thinking about ways we could create an innovative solution using AI. That's when we thought it would be cool if there existed an app which could change the language of any video we wanted in any language. That's how we came up with "Dub the Valley!"

The main inspiration was through brainstorming ideas around breaking the language barrier. We wanted to see how we can make comprehension in other languages more approachable. Through videos, we thought that it would be useful to find a way to translate it and make videos easily accessible for a global audience.

What it does

Our app allows users to upload any video of their choice and select any language they want from our dropdown options and the app translates the audio to whatever language the user selected. It works with 7 different languages so far. It transcribes the text and gets a transcript and then uses GCP Translate to translate it for us and then our code uses MoviePy to make it into a video with the dubbed audio.

This way, anyone can listen to their videos in other languages!

How we built it

We designed our logo using Photoshop, and for our website's UI/UX and prototype, we employed Figma. The current website version is following the Figma design, where additional features only exist in the prototype.

We used Python and Flask for the backend with the help of the Whisper library by OpenAI and Google Cloud Translate API for the translation. We also used HTML and CSS for the frontend part of our website. We also used Moviepy library to help us create dubbed videos quickly and efficiently.

Challenges we ran into

We ran into multiple obstacles while making this app. We ran into issues with the subtitles of our video not showing up when the video. We also ran into issues hosting our Flask app online as we are new to using the Google Cloud Platform. In this short period of time, making decisions for the prototype, the website itself and what features to implement were a major challenge we had to navigate around.

Another major obstacle was deciding what technologies to use as many technologies were not very feasible to implement.

Accomplishments that we're proud of

We're proud that we built such a cool app within the span of 36 hours and learned a lot about how OpenAI works and the things we can do with it. Also, coming from different backgrounds of studies and schools, it was really nice to see us all come together to make an interesting product.

What we learned

This is the first time our team and I dabbled deep with AI and learned how to create Flask apps using python. For some of us, this was our first time leaning python so it was a great experience overall.

What's next for Dub The Valley

In future iterations, we would love to add subtitles to our videos for better understanding and work our way to ship this to mobile. We would also like to implement the designs from the prototype as well as accessibility features (day/light mode), a chatbot, more languages and a history of past uploads.

Built With

Submitted to

Hack the Valley 8

Created by

I worked on the backend. Wrote the flask app with Whisper from OpenAI, MoviePy, Google Translate, Google Text-To-Speech

Yihoi Jung
I worked on the Front-end part of this website. I used HTML and CSS to built the user interface. This was my second hackathon, and it was really fun.

Chintan Patel
I worked as a UI/UX designer. I did research on our idea, created our designs and our prototype on Figma while using Photoshop too.

Rajsree Mandal
Fayaz Rafin
Computer Engineering Student with a passion for Software Development