Inspiration
What it does
How we built it
Inspiration
Millions of people navigate digital services in a language that isn't their first language. From booking medical appointments to registering for community events, even simple online forms can become barriers when someone struggles to understand the language.
I wanted to build something that removes that barrier.
SeeForMe is an AI accessibility assistant that helps people complete online forms through natural conversations in their preferred language.
The idea is simple: technology should adapt to people, not force people to adapt to technology.
What it does
SeeForMe is a conversational AI assistant built as a Safari extension for Mac.
Users connect their personal context, choose their preferred language, and open a web form.
SeeForMe can then:
- Understand personal information stored locally on the user's Mac.
- Communicate with the user in their preferred language through voice.
- Inspect web forms and identify the information required.
- Help complete forms in the language the form requires, even when that differs from the user's preferred language.
- Ask for missing information through natural conversation.
For example, a Bengali-speaking user can communicate with SeeForMe in Bengali while completing a form written in English or Japanese.
The goal is to make digital services more accessible, regardless of someone's language background.
How I built it
I built SeeForMe during the Sydney Mobile AI Hackathon using Apple's development ecosystem.
The application uses a Safari Web Extension to interact with web forms directly through their HTML and DOM structure, rather than relying on screenshots.
Personal context is stored locally on the Mac, allowing the assistant to access relevant user-provided information when completing forms.
For conversational interaction, I integrated ElevenLabs API for multilingual voice capabilities.
The system combines on-device browser interaction and context storage with cloud-powered voice and AI capabilities.
The project was developed using Xcode, Visual Studio Code, and AI-assisted coding tools.
Challenges I faced
One of the biggest challenges was building a working Safari extension within the limited hackathon timeframe.
Unlike a traditional chatbot, SeeForMe needs to understand real web forms, identify their fields, match information from personal context, and interact with the webpage.
Another challenge was multilingual voice support, particularly enabling conversations in Bengali while completing forms written in other languages.
I also had to balance on-device functionality with external AI services while keeping the experience simple and accessible.
What I learned
This project taught me how to combine Safari extensions, DOM-based browser interaction, local context storage, and conversational AI into a single accessibility-focused experience.
I also learned that building socially impactful AI isn't necessarily about creating the most complicated system. Sometimes, it's about making an everyday task accessible to someone who would otherwise struggle to complete it.
What's next
I want to continue developing SeeForMe into an accessibility tool that helps people independently navigate essential digital services.
The long-term vision is to support more languages, more complex forms, and more accessible interactions across healthcare, education, community services, and everyday life.
SeeForMe is built around one belief: language should never be a barrier to accessing essential services.
Log in or sign up for Devpost to join the conversation.