QA BOT AND BUGGY BANK

Inspiration

Modern UI testing tools rely heavily on fragile CSS selectors and static scripts. Even a small UI change such as renaming a class or reorganizing a button can break entire automation suites, forcing QA teams to constantly rewrite tests. This inefficiency wastes engineering time and slows down development cycles.

Our inspiration was to create a self-healing AI testing agent that understands intent instead of selectors. By combining natural language processing, visual UI understanding, and automated browser testing, we wanted to build a system that behaves like a human QA engineer — reading instructions, exploring the application, and identifying bugs autonomously.

The result is Autonomous QA Bot, an AI-powered UI testing agent capable of planning tests, executing them, discovering edge cases, and generating detailed bug reports automatically.

What it does

Autonomous QA Bot is an AI-driven testing platform that allows developers to test web applications using simple natural language instructions.

Instead of writing brittle automation scripts, users can describe their testing intent in plain English such as:

“Test the signup flow and verify password reset works.”

The system then:

Converts the request into a structured test plan.

Executes the test inside a real browser using Playwright.

Detects UI failures and unexpected behavior.

Automatically adapts if selectors change (self-healing).

Generates detailed bug reports with screenshots and logs.

Optionally exports issues directly to GitHub.

The platform also includes Chaos Mode, which automatically injects edge cases such as SQL injection attempts, extremely long inputs, invalid characters, and boundary values to discover hidden vulnerabilities.

To demonstrate the system, we created Buggy Bank, a deliberately flawed banking application containing multiple real-world UI bugs that the QA Bot can detect automatically.

How we built it

The system is built using a multi-layer AI-driven architecture designed for real-time automation and testing.

The frontend is developed with React, TypeScript, TailwindCSS, and Framer Motion, providing a modern dashboard where users can enter test instructions, watch live execution, and view generated reports.

The backend is built with Node.js and Express, which acts as the orchestration engine coordinating test planning, execution, reporting, and real-time communication.

AI-powered reasoning is implemented using Amazon Nova models:

Nova 2 Lite generates structured test plans from natural language instructions.

Nova Act interprets the UI visually and performs intelligent actions on web elements.

Actual browser automation is handled by Playwright, which runs the generated steps in a real browser environment.

Real-time progress updates are streamed through Socket.io, enabling the dashboard to display live execution steps and logs.

For data storage and report persistence, we use Firebase Firestore and Firebase Storage, while GitHub integration allows automatic creation of bug reports as issues.

Together, these components form a complete AI-powered autonomous testing pipeline.

Challenges we ran into

One of the biggest challenges was building reliable intent-based testing instead of selector-based automation. Traditional tools rely on static selectors, but our system needed to dynamically identify UI elements even when selectors changed.

Another major challenge was implementing self-healing behavior. The bot had to recover from broken selectors by analyzing visual text, ARIA labels, and semantic information from the DOM.

Handling natural language instructions was also complex. The system needed to parse vague human instructions and convert them into structured test steps that Playwright could execute reliably.

Real-time streaming presented additional challenges, especially synchronizing browser execution events with the dashboard using WebSockets.

Finally, designing Chaos Mode required careful logic to ensure edge-case inputs could stress-test applications without breaking the automation workflow itself.

Accomplishments that we're proud of

We successfully built a fully autonomous AI testing agent capable of planning, executing, and reporting UI tests without manual scripting.

Key achievements include:

Natural language test generation powered by AI

A self-healing automation system that adapts to UI changes

Real-time live test execution streaming

Automated bug reporting with screenshots and logs

One-click GitHub issue generation

Chaos Mode for automatic edge-case discovery

We also developed Buggy Bank, a demo banking application containing 18 intentionally injected bugs, allowing the QA Bot to demonstrate its ability to detect real-world UI issues automatically.

The entire system works end-to-end, from intent → execution → bug discovery → report generation.

What we learned

Building Autonomous QA Bot taught us several important lessons about AI agents, browser automation, and system orchestration.

We learned how AI models can transform traditional automation workflows by focusing on intent understanding rather than rigid scripting.

The project also deepened our understanding of real-time systems, particularly how WebSockets enable live feedback between backend automation engines and frontend dashboards.

Integrating AI planning with browser automation revealed how powerful agent-based architectures can be when multiple systems collaborate.

Additionally, developing Chaos Mode demonstrated the importance of security and edge-case testing in modern web applications.

Overall, the project showed how AI can fundamentally change the way software testing is performed.

What's next for QA BOT AND BUGGY BANK

The next step for Autonomous QA Bot is to evolve into a fully autonomous QA engineer agent capable of testing large production applications continuously.

Future improvements include:

Integration with CI/CD pipelines for automated regression testing

Support for mobile and cross-browser testing

Advanced AI reasoning for multi-page workflows

Visual UI comparison and layout regression detection

Automatic prioritization of critical bugs

Integration with platforms like Jira, Slack, and GitHub Actions

For Buggy Bank, we plan to expand the demo with more realistic scenarios such as payment systems, authentication flows, and API-level vulnerabilities.

Our long-term vision is to build a self-improving AI testing platform