Inspiration
We felt that many good games made outside of the US have never been translated to English. This is meant to not only translate games for people that want to play foreign games, but also learn a new language while playing. By using Fish Audio, we grant users one of the crucial parts of human expression: emotion.
What it does
It translates a game completely live and synthesizes audio for auditory learners.
How we built it
We utilized cv2 for screen capture and Tesseract-OCR to extract the text and translate the game with DeepL API.
Challenges we ran into
Implementing Fish Audio with the the translation system and making sure Tesseract had good accuracy in extracting text
Accomplishments that we're proud of
Being able to create a live caption and translation in the game without causing significant issues with latency.
What we learned
How to utilize Tesseract-OCR, Fish Audio, Pyside6, and the Facial Emotion Recognition (FER) python library. We also learned the importance of adapting quickly when developing (especially within a 24-hour timeframe)!
What's next for GameTractor
We would need more variations in voices for Fish Audio, more accurate text extraction, and potentially replacing Tesseract-OCR with ML translation instead.
Log in or sign up for Devpost to join the conversation.