Project Story
Inspiration
Our journey began with a recognition of the substantial gap that language barriers create, often leading to miscommunication and exclusion. Inspired by the idea that mastering a new language can significantly bridge this gap and foster inclusivity, we turned to a universally loved medium—movies. The immersive nature of films accelerates comprehension and retention of new languages. With Gemini AI, we saw an opportunity to harness cutting-edge technology to develop tailored educational content, making learning not only effective but also enjoyable.
What it does
Our project cooks up a recipe for "mind food" by utilizing movie subtitles as a base ingredient 100% automated. It processes these subtitles to extract relevant scenes and overlays them with educational voiceovers of a text generated by Gemini AI, both before and after each scene. This includes an engaging intro and a thought-provoking question to conclude. The final product is a seamless combination of movie clips, transformed into a memorable and engaging language learning experience.
How we built it
We assembled a team of diverse AIs to tackle various aspects of content creation. These AIs parsed created movie subtitles for Gemeni teacher and it's students, identified relevant phrases, and segmented movie scenes accordingly. Gemini AI played a pivotal role, crafting educational text and generating speech overlays. We employed text-to-speech technology to vocalize the content, ensuring a natural and accessible learning format.
Challenges we ran into
One challenge was integrating text in both English and German without compromising the speech quality. We (Gemeni and I) innovated a solution by tagging German phrases specifically, allowing for precise language handling in voice synthesis. Automating the entire process posed several technical hurdles, including scene extraction, content description via Gemini Vision Pro, and dynamic subtitle integration. Despite these challenges, Gemini's advanced capabilities facilitated contextual understanding, enhancing the overall content quality!
Accomplishments that we're proud of
We're particularly proud of transforming underutilized movie scenes into valuable educational tools. Our YouTube channel, German Popcorn showcases these creations and has begun to gather an enthusiastic community of learners who appreciate the quality and approachability of our content.
What we learned
This project was a profound learning experience in leveraging AI to break down traditional educational barriers. We discovered the versatility of Gemini AI in creating customized educational narratives and its potential to revolutionize not just language learning but various forms of digital education.
What's next
Looking ahead, we aim to expand the scope of our project to include a broader range of educational videos. We envision using Gemini AI to craft learning materials across different subjects, leveraging our automation script to maximize content production without sacrificing quality.
Built With
- elevenlabs
- gemeni
- python
- typescript
- whisper