Inspiration
Videos are made to help people learn another language by injecting the learning into a popular foreign show.
What it does
Simply provide a video link in a foreign language. LinguaMedia will parse the content, extract meaningful words, generate subtitles and word bubbles, and help you practice newly learned vocabulary in a post-learning session and quiz section!
As the user watches the video, LinguaMedia will parse it in real-time, generating subtitles and popping up keyword chat along the way. Once the video finishes, a review section will display detailed information about each vocabulary word, along with pronunciation audio. This will be followed by a quiz section to reinforce learning.
How we built it
The app front-end backend is built with Relex.dev The backend logic/task is modulized and chained with fetch.ai and uagent framework.
- Video transcription is captured using Python script.
- Transcripts are processed using the GROQ model to extract the meaningful vocabulary.
- Translation, Description, Pronunciation text, and Quiz content are generated with the Google Gemini model.
- Pronunciation Audio is generated with Cartesia API
Challenges we ran into
new framework and technology and documentations to read and learn
Accomplishments that we're proud of
It works functionally, and the app has components.
What we learned
Modulize a process to break down big problems.
What's next for LinguaMedia
Fully link front-end and backend logics for full experience.
Log in or sign up for Devpost to join the conversation.