AltGen

• 

  AltGen logo: blue-purple icon with mountains, a sun, and sparkles, plus text “AltGen – An invisible assistant for everyone.”

Inspiration

When people hear the term "visual impairment," they often think of complete blindness. However, visual impairment exists across a wide spectrum, from various forms of low vision to total blindness. According to the World Health Organization (WHO) in 2020, approximately 300 million people worldwide live with severe visual impairment, including blindness.

However, the web environment we interact with daily presents an "invisible barrier" to people with these impairments. Many websites feature inaccessible images, and even when image labels (alt text) are present, they are often extremely vague or overly simplistic.

What It Does

AltGen is a Chrome extension that utilizes Chrome's Multimodal Prompt API. It aims to generate audio descriptions for images for visually impaired users directly on-device, without transmitting any personal data.

How We Built It

The AltGen project began with a single question: "How can we improve web accessibility while perfectly protecting user privacy?"

We identified a core problem: most AI image description models require sending image data to a server. This inherently carries the risk of exposing a user's private browsing history.

While searching for a solution, we discovered the potential of Chrome's built-in "Multimodal Prompt API." This API enables on-device AI processing, allowing images to be analyzed directly within the user's machine without ever being sent externally.

Our biggest challenge was leveraging this new API to build an engine that generated rich, contextual "audio descriptions," not just simple labels like "cat." We needed to describe the scene and context.

Accomplishments That We’re Proud Of

• Implemented On-Device Processing: We completed the core logic that generates image descriptions directly on the user's device using Chrome's Multimodal API, without needing a server.
• More Detailed Image Descriptions: Leveraging the powerful multimodal capabilities of Gemini Nano, it provides detailed and rich image descriptions that are easy for anyone to understand.

Built With

• react
• tailwindcss
• typescript
• wxt