InkSwap

Manga that speaks your language.

Comment

Inspiration

InkSwap came from a frustration I've had as a manga reader for a long time.

I once followed a manga series I absolutely loved, only for the group translating it into English to eventually drop it. The original Japanese manga (the "raws") kept getting new chapters, but there was suddenly no English version to read.

Eventually, fan translators tried to pick it up, but the translations were difficult to follow. Technically, the words were in English, but they felt like machine-translated English: literal translations that didn't always capture what the characters were actually trying to say.

And that wasn't the only problem.

If you follow ongoing manga, you've probably experienced the painful gap between the raws and available translations. The original release might already be on Chapter 80 while the translation you're following is still on Chapter 40. When you've been left on a cliffhanger, waiting for dozens of chapters to be translated can feel impossible.

I've gotten impatient enough before to open the Japanese raws myself and try to follow the story from the artwork alone. I couldn't understand the dialogue, of course, but it always made me think: what if I could just translate the manga right here?

Not by copying text into another app. Not by downloading pages and uploading them somewhere else. I wanted to open the original manga, switch something on, and have natural English appear directly inside the speech bubbles as I read.

It was an idea I'd had in the back of my mind for a long time. I just didn't know how I would build it.

This hackathon gave me the reason to finally try.

What it does

InkSwap is a live manga translation Chrome extension.

You open a manga in your browser, switch InkSwap on, and keep reading. It detects the speech bubbles on the page, understands the text inside them, and places the translation directly over the original bubble.

More importantly, InkSwap aims to translate meaning, not just words. Instead of producing awkward word-for-word translations, it tries to preserve what the character is actually saying—the tone, intent, and nuances that make dialogue feel natural.

And it happens right where you're already reading.

As you move through the manga, InkSwap continues translating new pages. Move to the next chapter, and it keeps going without requiring you to turn it on again. Previously translated content can also be saved and reused.

InkSwap can work with multiple source languages and currently translates into English, Spanish, and French.

For this proof of concept, I tested InkSwap on MangaDex and Shonen Jump+, including different reader layouts and Shonen Jump+'s two-page view.

The idea behind InkSwap is simple: you shouldn't have to leave the manga to understand the manga.

Open the raws. Switch it on. Keep reading.

How we built it

Interestingly, I didn't start this hackathon with InkSwap in mind.

I began by installing the Devpost Learn Skill Pack provided in the hackathon instructions. Since I didn't have an idea yet, I started with brainstorming. The agent asked questions about my interests and problems I had encountered, and during that conversation I remembered an idea I'd had for a long time: being able to translate raw manga live while reading it.

I'd always assumed it would be incredibly difficult to build. But as I explored the idea with the AI, I started to see how it could actually work.

From there, I used the AI as a development partner throughout the process. We narrowed the scope, defined what a successful proof of concept should look like, wrote the requirements and specifications, divided development into phases, and tested each phase before moving forward. When something didn't work, I would bring the result back, reason through what was going wrong, make changes, and test again.

InkSwap itself is built as a Chrome extension. It captures the manga pages in the browser and uses Claude to understand and translate the dialogue, while the extension handles detecting and mapping the speech bubbles so that each translation can be placed back over the correct location on the page.

One thing I specifically wanted to learn during this hackathon was how to write better prompts myself. I wrote the initial translation prompt and continued refining it as I tested InkSwap on real manga pages. With guidance from the AI, I adjusted the prompt to better handle things like natural translations, speech bubbles, names, honorifics, unclear text, different source languages, and the nuances of manga dialogue.

That iterative process—define, build, test, inspect, refine—became a major part of how InkSwap was built.

Challenges we ran into

One of the biggest challenges was getting translations to actually belong to the correct speech bubbles.

Early on, InkSwap could take a screenshot of the manga page and identify text, but placing the translated text correctly was another problem entirely. At one point, English translations were appearing vertically as individual characters instead of naturally filling the bubble.

It took a lot of testing and iteration to develop a better approach: first identify and number each speech bubble, then map the corresponding translation back to that specific bubble. That eventually gave us the much more accurate one-to-one bubble matching seen in the current version.

And then there were the wonderfully strange edge cases.

At one point, Spanish source pages kept returning empty translations. The request was telling Claude that the page was Japanese, so when I tested InkSwap on a Spanish MangaDex chapter and asked it to translate into English, it sometimes returned nothing. Adding one sentence to account for other source languages fixed the problem.

In another test, a speech bubble that was partially cut off by the edge of the page image wasn't detected correctly. InkSwap fell back to Claude's approximate position for the translation—and that position happened to be directly over a character's face. So, for a brief moment, one of my characters was wearing the translation instead of saying it.

Different manga sites, page layouts, bubble shapes, and languages exposed different assumptions that didn't hold up once the extension encountered real pages.

Each of those failures became part of the development loop. I would inspect what happened, discuss possible causes and solutions with the AI, adjust the implementation or prompt, and test again.

The biggest lesson was that getting an AI to translate manga wasn't really the hardest part. The challenge was getting all the pieces to work together reliably: understanding the page, identifying the right bubbles, producing a natural translation, mapping each translation back to the correct location, and making the final result feel like it actually belongs on the manga page.

There were plenty of moments where InkSwap produced something completely different from what I expected, but each one helped refine the approach.

And eventually, the idea I'd kept in the back of my mind because it seemed almost impossible became a working proof of concept.

Accomplishments that we're proud of

The moment I'm most proud of was surprisingly simple: I read a whole chapter.

Once InkSwap was working end to end, I went straight to a raw manga site, switched the extension on, and started reading. I made it through an entire chapter smoothly, with the translations appearing directly in the speech bubbles as I went.

I cannot describe how exciting that was.

What made it even more meaningful was that InkSwap wasn't actually my first attempt at this idea. Some time ago, I tried building something similar using OCR, but I couldn't get it to work the way I imagined. Eventually, I got frustrated and dropped the project.

So when I watched InkSwap identify the bubbles, translate the dialogue, place it back onto the page, and continue working as I read, I almost couldn't believe it. Something I'd previously written off as too difficult had become something I could actually use.

Beyond simply getting it to work, I'm proud of how much of the reading experience came together: meaning-based translations, one-to-one bubble matching, support for different target languages, continuous translation between pages and chapters, saved translations, and support for different manga reader layouts.

But ultimately, my favorite test was also the simplest one: could I actually sit down and enjoy a raw manga with it?

And I could.

What we learned

One of my biggest lessons from building InkSwap was the importance of doing the thinking before jumping into implementation.

Taking the time to define the requirements, non-functional requirements, PRD, scope, and expected behavior gave the project structure. It also taught me how valuable it is to ask about the "unknown unknowns" early—questions and edge cases you might not immediately realize you need to consider.

It's much better to discover those questions while defining the project than halfway through building it.

I also learned a lot about working effectively with AI during development. Instead of treating AI as something I could simply give an idea to and expect a finished product from, I learned to work iteratively: define what I want, question assumptions, test the result, identify what went wrong, refine the approach, and try again.

Prompt writing became a big part of that learning process as well. Small decisions about how the translation prompt described context, tone, speech bubbles, uncertainty, names, and honorifics could noticeably change the results.

More than anything, this project taught me that an idea feeling technically intimidating doesn't necessarily mean it isn't worth attempting. Sometimes you just need a better way to break the problem down.

What's next for InkSwap

If I continue developing InkSwap, one of my first goals would be supporting more manga sites.

Different sites have different readers and ways of rendering their pages, so I'd love to test InkSwap against more of them and explore how the extension can adapt to those different reading experiences.

I'd also like to expand the number of supported target languages beyond English, Spanish, and French, while continuing to improve how naturally translations preserve the meaning and personality of the original dialogue.

Another major area I'd explore is reducing AI costs. For a tool designed to continuously translate pages while someone reads, cost can add up quickly. Finding smarter ways to reuse translations, avoid unnecessary processing, and reduce the amount of AI computation required would be important for taking InkSwap beyond a proof of concept.

And I think the idea could eventually extend beyond the Chrome extension itself.

Imagine official raw manga platforms being able to offer this kind of translation directly to international readers. Instead of waiting for a separate localized release—or turning to unofficial copies—someone could subscribe to the original platform and read the manga in a language they understand.

Maybe that means partnering with existing manga platforms. Maybe one day it means building a manga reader designed around this experience from the beginning.

Either way, that's the direction that excites me most: making it possible for more people to support and read manga from its official source, regardless of the language it was originally published in.

Language shouldn't be the reason someone can't enjoy a great manga.

Built With

Share this project:

Updates

Submission history