Clueless AI

• 

  Clueless AI

Inspiration

We were inspired by the universal experience of frustration when navigating new or poorly designed websites. Whether it's a family member struggling with an online banking portal or a new employee learning a complex internal dashboard, people often need a little guidance. We wanted to create a tool that acts as an intelligent, on-demand guide, making the web more accessible and less intimidating for everyone.

What it does

Clueless AI is a browser extension that provides real-time, interactive help on any website. When a user gets stuck, they can open our extension and type a simple request in plain English, such as "find my order history" or "create a new project." Our AI then does two things simultaneously: Voice Guidance: It provides clear, spoken instructions for each step. Visual Walkthrough: It dims the screen and highlights the exact buttons, links, or form fields the user needs to interact with, guiding them through the task from start to finish.

How we built it

We built Clueless AI as a lightweight and fast browser extension using core web technologies. Frontend & Core Logic: We used HTML, CSS, and JavaScript to build the user interface and the extension's framework. Visual Guidance: The on-page highlighting and step-by-step tours are powered by the flexible JavaScript library, Shepherd.js. Voice Output: For the voice narration, we leveraged the native Web Speech API built into modern browsers, which allowed for rapid implementation without external dependencies. AI & NLP: To understand user requests, we implemented a client-side NLP system using a JavaScript library to parse user intent and identify keywords. This allows us to dynamically find the correct elements on the webpage to guide the user toward.

Challenges we ran into

The biggest challenge was universality. Every website is built differently, so creating a system that can reliably find the "login button" or "search bar" across countless HTML structures was complex. We had to move beyond simple element selectors and develop a more intelligent "finder" function that looks for clues like ARIA labels, placeholder text, and common naming conventions to locate the correct target.

Accomplishments that we're proud of

We are incredibly proud that we successfully created a working end-to-end prototype within the time constraints of this hackathon. Our tool can be installed in a browser and immediately provide value on different websites. The dynamic element-finding system, while still early, is a major accomplishment and proves the core concept is viable. Seeing the visual guide and hearing the voice kick in based on a typed command for the first time was a huge success for our team.

What we learned

This project was a deep dive into the architecture of browser extensions, from manifest files and content scripts to messaging between different parts of the extension. We learned how to programmatically interact with and modify live webpages in a clean, non-intrusive way. Furthermore, we gained valuable experience in implementing a practical, client-side AI that solves a real-world usability problem.

What's next for Clueless AI

The future for Clueless AI is bright. Our next steps would be to: Improve the AI: Integrate a more sophisticated NLP model to understand more complex, multi-step commands. Introduce "Learning Mode": Allow users to "teach" Clueless AI how to perform a task on a new site, which it can then save and repeat for them later. Expand Accessibility: Enhance the tool with more features specifically designed to help users with disabilities navigate the web more easily. Team & Enterprise Onboarding: Position Clueless AI as a tool for businesses to create interactive tutorials for their own software, simplifying customer and employee onboarding.

Built With

• browserextensionapi
• css
• html
• javascript
• manifestv3
• nlp.js
• serverless
• shepherd.js
• websearchapi