VORA (Voice Operated Response Agent)

Inspiration

Think back to the last time you emailed your doctor, checked your bank account, or just read the news. It might seem like a trivial task for you and I, but for over 50 million people worldwide, tasks like these are nearly impossible by themselves.

Computers were built for those who can point and click, but for people who have diseases such as Parkinson’s, ALS, or carpal tunnel, interacting with devices has become a significant challenge, especially in our increasingly digital world. They need tools that bridge the gap. That's why we built Vora, a voice-first way to explore the internet.

What it does

Vora is a Chrome extension that lets you interact with the web through voice commands. It can open your email, check your bank balance, open Netflix, and do everything else in between! If you want something done, simply speak it, and Vora will make it happen.

How we built it

The Vora extension has three main parts: a live DOM parser, a service worker for API calls, and a side panel. The side panel listens for commands, once it hears one, the DOM parses extracts the most important elements of the page and sends that + the voice command to the service worker. The service worker then makes a call to Claude, and gets back a BrowserAction (i.e. click, type, navigate, scroll, submit) which gets executed in the user's browser. In the case of a destructive action like a payment or deletion, a spoken confirmation is always required before anything runs.

The project itself was built during the Calpoly SLO Kiro Hackathon. Both the Kiro IDE and CLI were used for idea validation & coding.

Challenges we ran into

A notable challenge we experienced was dealing with the accuracy of the google chrome speech-to-text api. While it worked well for simple tasks, exact wording or context was often incorrect or cut off which led to issues navigating action-heavy sites. The solution was to utilize a hybrid pipeline of google speech to text and Claude for filling in missing context and command correction.

Accomplishments that we're proud of

We’re extremely proud we were able to ship a fully working voice-driven Chrome extension in under 24 hours, especially as a team of only 3 people. Vora requires minimal setup, handles 18 distinct browser actions, and works across all websites.

What we learned

Kiro, being the spec-driven agent that it is, made us reconsider how to structure a project before we even started building. We realized that because it’s spec driven, it works the best when given parameters and a goal. This goes just beyond Kiro though, in the ever increasing world of AI coding, context becomes more and more important for any agent. This Kiro hackathon taught us just how true that really is, and we’re excited to use these new discoveries on our next projects.

What's next for VORA (Voice Operated Response Agent)

Our next steps are to incorporate more accessibility features to Vora, such as canvas-rendered app support for apps like Google Docs and Figma, and multi-step command chaining to make our services even more efficient. We truly believe Vora is a step toward a more accessible internet, and we are excited to keep building for the people who need it most.