Blindly

😮 Inspiration

For screen reader users and those who are visually impaired, the web is often uncertain and risky. Traditional screen readers like NVDA or VoiceOver tell users what a button says — but not what will happen if they press it.

Will it:

Charge a credit card?
Submit a government form?
Change CPP (Canadian Pension Plan) details?
Delete an account permanently?

This uncertainty forces users to slow down, guess outcomes, or avoid actions altogether.

We realized accessibility isn’t just about reading content — it’s about understanding consequences before acting. That insight led us to build Blindly.

💻 What It Does

Blindly is a conversational, AI-powered screen reader companion that provides awareness before action. It predicts what will happen when users interact with websites before they click, turning web navigation into a guided, confidence-building experience. Blindly also allows you to interact with the webpage in anyway, filling out forms or even browsing through the internet.

🔍 Intent-Aware Actions

Explains what an action actually does
Predicts what happens next
Assigns a clear risk level (LOW / MEDIUM / HIGH)

🗣️ Conversational AI Companion

Blindly speaks using a natural human voice
Users respond entirely by voice
Guides users step-by-step through forms and tasks

✍️ Assisted Form Filling + Web Browsing

Asks one question at a time (e.g., “What is your postal code?”)
Confirms responses before filling fields
Never auto-fills without explicit user approval

Blindly turns web navigation from a guessing game into a conversation.

🍁 Canadian Focus

Blindly is built specifically with Canadian users in mind.

We created a repository of 100+ Canadian healthcare, financial, and identity-related keywords to detect high-risk actions unique to Canadian systems.

Blindly can:

Detect Interac e-Transfers and warn that they are difficult to reverse
Recognize Canadian government websites (canada.ca, ontario.ca, CRA)
Identify high-risk forms involving CPP, OSAP, and CRA
Validate Canadian postal codes

This ensures Blindly doesn’t just work globally — it works correctly for Canadians.

🎨 User Experience & Accessibility

User experience was a core focus in every feature we built.

Fully keyboard-accessible with four custom hotkeys
Tab-based navigation through all interactive elements
Automatic voice feedback on hover or focus
High-contrast UI compliant with WCAG AAA standards
Voice-first interaction
Webpage summaries
Ability to let Blindly AI Companion handle webpage browsing and formfilling

Blindly isn’t just accessible for screen reader users — it’s accessible to use.

🧠 How We Used Gemini

We intentionally did not rely on AI alone.

Blindly uses a layered hybrid approach:

Local Analysis and Risk Analysis (Fast & Deterministic)

DOM properties (tag, type, text)
Form structure and workflow signals
Keyword detection, including Canadian-specific patterns

Gemini AI Reasoning (When Needed)

Gemini is only invoked for ambiguous or high-risk actions (identified in the Risk Analysis). It is provided with DOM context, detected signals, and page summaries to convert technical information into clear, human explanations.

This keeps the system fast, explainable, and trustworthy.

🔊 Voice & Interaction

Speech-to-Text via the Web Speech API
Gemini for intent and workflow understanding
ElevenLabs for natural, human voice output
Web Speech TTS fallback for reliability

Because Blindly can safely manipulate the DOM, users can fill out forms, search the web, read articles, and navigate websites entirely through conversation.

Challenges We Faced

Predicting intent without clicking required analyzing workflows and page context
Balancing latency and reliability meant prioritizing local analysis over AI calls
Privacy concerns led us to implement local redaction of sensitive data
Building an accessibility tool that is itself accessible required full keyboard and screen reader compatibility

What’s Next for Blindly

Firefox and Edge support
Improved voice personalization
Smarter workflow prediction across pages
OCR for image-based buttons
Multi-language support (including French)
Deeper integration with form builders (PDFS, Excel, etc.)
A developer platform for intent signals
Community-driven detection patterns
Collaboration with accessibility organizations

Blindly shifts accessibility from reading to understanding — because real accessibility isn’t about what’s on the screen, it’s about knowing what will happen next.