Image Translate

Inspiration

Language translation is essential when traveling to a different place. Translation is especially necessary in museums or in religious places where the description of an art piece, display item or a saint may not be in English. Hence, Image translation.

What it does

Image translation extracts the text from an image that is uploaded from the computer or is captured by taking a picture. The extension then identifies the language with a built-in chrome language identification tool and translates the text to English.

How we built it

The "Image translate" application is a chrome extension that allows the user to upload an existing image from the device or take a picture of the image that has text. The app then extracts the text using OpenAI and then uses chrome built-in language detection tool to identify the language of the text. The app takes the first result that has the highest matching rate from the identified list of languages and translates the text into English. If there is no text to translate or the text is already in English then the extension just states that there is no text to translate or just reiterates the same text in English. The extension uses built in prompt Api to translate the text into English pretending to be an eloquent linguist.

Challenges we ran into

There were many challenges that I ran into as most of the features like language translation is not available in Chrome as Chrome canary is needed to check the functionality of built-in language detection feature. Also, the extension is modified and workarounds are included to make the extension available in both Chrome and also in Chrome canary.

Accomplishments that we're proud of and earnings

1. Implementing and understand the importance of Chrome built-in language detection capability without the need of a cloud server to send and receive the language data decreasing latency and server costs.
2. Download new Chome canary browser with built-in features.
3. Signing in for the preview program and implementing the language translation feature.

What's next for Image Translate

Using Chrome built-in features entirely to detect and translate the language without using any other AI tools thereby reducing server costs and decreasing latency.

Built With

• amazon-web-services
• aws-lambda
• chrome
• chrome-canary
• google-built-in-ai-challenge
• gpt-4o-mini
• openai