Project name Wayfinder
Elevator pitch Wayfinder turns any webpage into a few spoken choices you can answer by voice, tap, or Braille, without picking a disability or learning the site's layout.
About the project
Inspiration Websites still make people learn each site's layout. Wayfinder reverses that. It compiles a page into one semantic state: what this is, what matters, and a handful of next actions. The same state is spoken, shown, and rendered as Grade 1 Braille. You do not pick a disability first. Whichever channel you use is the interface.
What we learned A checker that says "Verification: SUCCESS" is not an interface. People need a phone-menu conversation: it speaks, and a tap or a sentence interrupts it. The visual page stays usable the whole time. Braille and a motor cue are an extra sensation alongside the page, like bass at a concert, not a separate mode.
How we built it The Mac app is Python and FastAPI. Playwright reads a real page, including any http(s) URL, and a labeled Harbor Line fixture works offline. Choices are grounded in the page. A consequential submit waits for confirmation. Optional pieces stay off unless configured: ASI:One or local Ollama for summary wording, Sentry for errors and traces, and a separate Fetch.ai Reader uAgent. The web UI uses the browser speech APIs and a laptop-camera motion control. ESP32 firmware is scaffolding for a later bench, not a claimed live motor.
Challenges Speech recognition and the camera are easy to over-trigger, and browsers block the mic until a gesture. We kept the command box on the same path as speech so a blocked mic still works. Promo credits for ASI:One were expired, so the model path falls back to local Ollama and then to a deterministic reader. Hardware was not flashed during the hackathon.
Built-with tags Python, FastAPI, Playwright, JavaScript, Web Speech API, Sentry, Ollama, Fetch.ai, ESP32. Skip any tag the form rejects.
Try-it / GitHub https://github.com/StanchPillow55/Wayfinder
Do not add a demo URL, YouTube URL, or Vimeo URL. If the video field is required and empty, leave it empty.
Team
Email: [email protected], only if that address is the one shown on the account. If the account shows a different email, use that one. Name: Bradley Haraguchi if the account name matches; otherwise the name on the account. No other teammates. School: San Jose State University if the email is @sjsu.edu. Check the two required "I understand" boxes about email/repo and being present for judging. Sponsor challenges
Agentverse - Fetch.ai Sentry Do not select Browser Use, Visa, ElevenLabs, Espressif, or other prizes.
Event track One track only. Choose the accessible / human-interface option if it is there. If the options are unclear, choose the most general participant track.
AI tools
Generative AI: Yes Check Llama, and Other if there is a text field (ASI:One). Answer: Optional ASI:One (asi1-mini) or local Llama 3.1 rewrites the page summary and the order of grounded choices. The model cannot invent buttons. If the key is missing or the call fails, Wayfinder uses the deterministic reader. The demo runs with the model turned off. Gemini project number: leave blank. ASI:One user handle: leave blank unless you can see the real handle on an open asi1.ai tab. Do not invent one. GoDaddy domains Leave blank.
Feedback Sentry traces and the Fetch.ai uAgent were useful optional modules, and the core demo does not depend on them.
Draft URL, if you still have the tab: devpost.com/submit-to/30323-ai-hackat…inder/additional-info/edit
Log in or sign up for Devpost to join the conversation.