Simul

Browse the internet in your native language while interacting with the original website.

Comment

Inspiration

Anyone that's traveled or lived in a foreign country has used a built-in browser or translation tools to find that they don't work. The web pages often break and you have to switch back and forth between the languages just to fill out simple forms. Frustrated with having to switch the language I'm viewing copy and paste between forms. I try to remember exactly where on a form I needed to click the correct drop-down. I decided to make a tool to help me along.

What it does

In a side panel or a pop-out window, you can see a real time copy of the web page that you're viewing in the language of your choice. Everything runs locally so you don't have to worry about sending your private data somewhere else for processing. We are using Chrome's built-in translation tool API, which requires Chrome 138, and we are using Tesseract, which is a fairly old tool for OCR.

How we built it

One very long session of codecs using the BMAD framework. We started with a very basic replica of the web page that translated a few items and then quickly iterated and built from there.

Challenges we ran into

Extracting text from images from many different languages is very difficult. There are some tools that can extract for specific languages but we didn't see any that can natively just pull text out of any image. Free on-device tools are still limited. Many pages update in weird ways and so reading the DOM and trying to replicate that caused a lot of problems.

Accomplishments that we're proud of

We're able to quickly build and release a working product that was useful to us on day one. We've been able to quickly iterate and add and fix popular broken websites. I'm happy we were able to find a way to run everything on device.

What we learned

We tried to push the boundary by minimizing how much intent and guidance was given to the AI to see how far we could get. We learned that it's much better than it was before and we can quickly get to a working product with little feedback provided but ultimately we're still the place where we need to write out detailed specs.

What's next for Simul

It will be available as an open source project for anyone to fork or contribute to. We'd like to continue iterating on the project, fixing bugs, and improving what can be done on device. Evaluate what we can find for other available packages that might be able to support or contribute to the analysis. There is a lot of room for improvement in the OCR and pulling text out of images to be translated. We would like to add the option to connect to other AI backends and use your existing models for higher quality translation.

Built With

  • 5.6
  • bmad
  • codex
  • sol
  • ultra
Share this project:

Updates