-
Raw manga page uploaded into app
-
Translated manga page received in app
-
Example of auto-translated page
Inspiration
We often want to read certain manga because we are interested in their stories, but are completely barred from doing so purely because there is no translation for them in English. As a result, we decided to make our own solution, in which we take the pages of the manga, translate them, and rewrite them for you.
What it does
It takes the image of the manga page and sends it to a backend. The backend first uses a neural network trained with thousands of cases of actual manga to detect the areas of writing and text on the manga. Then, the server clears that area out of the image. Using Google Cloud Services, we then take the written Japanese and translate into English. Lastly we rewrite that English in its corresponding postitions on the original image to complete the manga page.
How we built it
We used python and flask with a bit of html and css for the front end Web server. We used Expo to create a mobile front end as well. We wrote the backend in python.
Challenges we ran into
One of the challenges was properly using Expo, a service/platform new to us, to fit our many needs. There were some functionalities we wanted that Expo didn't have. However, we found manual work-arounds.
Accomplishments that we're proud of
We are proud of successfully creating this project, especially because it was a difficult task. The fact that we successfully completed a working product that we can consider using ourselves makes this accomplishment even better.
What we learned
We learned a lot about how to use Expo, since it was our first time using it. We also learned about how to modify images through python along the way.
What's next for Kero
Kero's front-end can be expanded to look nicer and have more functionality, like multiple images at once for translating a whole chapter of a manga.
Log in or sign up for Devpost to join the conversation.