Mangime

slide_3
slide_1
slide_2
slide_4

Inspiration

A friend of ours always complains that he can't read manga in his language and wishes he could experience his favorite stories in more engaging formats, like animation or interactive scenes. This inspired us to create a solution that brings manga to fans worldwide in multiple languages and formats.

What it does

We transform traditional comic books into animated, translated, and interactive experiences. Comic writers will upload their comics, and our platform automates translation, animation, and format customization, allowing fans to enjoy manga as a comic, animation, or even speak to the characters in their own language. Specific features are:

Downloads manga images from Kakao webtoons
Generates scene descriptions and dialogue using OpenAI GPT-4 Vision
Creates audio narration using ElevenLabs TTS
Produces smooth transitions between frames using Luma AI
Caches all intermediate results for faster subsequent runs
Supports processing a specific number of frames or entire chapters

How we built it

We use openAI to identify text dialogues in comic books and translating to any language. We use 11Elevenlabs to narrate the comic book in different languages, and the conversational AI feature to talk to the comicbook characters We use lumaLabs to animate the narration of the comic book

Challenges we ran into

We had a different idea. We tried to connect the conversational ai feature to whatsapp. However, we couldn´t hack into the Audio and Video interfaces to intercept whatsapp web input and output streams in our macs. We think we can do this if we develop a new driver that could be installed natively but we need more time for this. Yesterday at 8pm we changed the idea.