QARun.ai - AI testing

AI powered testing, self healing, self writing, built with bolt.new

Comment

Inspiration

We realized something fundamental: developers spend 20–40% of their time actually coding, while the rest is swallowed by writing tests, fixing bugs, managing CI/CD, and dealing with flaky builds. Testing alone can consume up to third of the software lifecycle effort not to mention debugging efforts afterwards. Our mission for QARun.AI was born: reclaim developer flow by automating test creation and execution through cutting-edge AI in a way that adapts and handles traditional and new ways of development..

What QARun.ai does

QARun.AI is a next-generation platform that reimagines software testing. Unlike traditional automated testing tools, QARun.AI uses advanced AI to approach testing like a knowledgeable manual tester—but with even more capability, speed, scaleability and reliability. QARun.AI can see and understand what’s on the screen, adapt to changes and heal broken tests.

Key Features

  • AI-Driven Test Creation: Upload your codebase and provide a general description. QARun.AI instantly generates industry-standard test cases and detailed step definitions, making test authoring nearly effortless.
  • Smarter Test Execution: QARun.AI “understands” the test intent and the user experience. Tests are robust and flexible—not brittle—so even if you make minor updates to your solution, QARun.AI can adapt and continue testing without breaking.
  • Self-Healing & Auto-Improvement: When changes are detected or tests fail due to minor UI or logic differences, QARun.AI automatically heals the tests and suggests further improvements, reducing maintenance to a minimum.
  • Intelligent Failure Analysis: On failure, QARun.AI delivers a comprehensive log and a plain-language summary of exactly what went wrong, so you can resolve issues quickly and confidently.
  • Batch Test Execution & Coverage: Easily run and re-run batches of tests at the click of a button, ensuring maximum test coverage for every build and deployment.
  • Multi-Language & Global Support: Supports multiple programming languages and enables global teams to collaborate on testing seamlessly.
  • Fully Responsive UI: Manage your testing workflows from anywhere with a modern, responsive web interface—desktop, tablet, or mobile.
  • Deep Bolt.new Integration: QARun.AI extends the Bolt.new GitHub version, adding a “Test” button. With a single click, you can generate and run tests directly alongside your coding process, all within Bolt.new.

How we built it

Architecture We started by building our core platform within Bolt.new building out core journeys and UI. Bolt.new made it effortless to add in features like batching, overview reporting screens and subscription handling. From there, we integrated the AWS SDK to provision and manage back-end infrastructure in the cloud in a nodejs backend —spinning up test runners and handling execution environments on demand.

For AI-powered test generation, we leveraged Anthropic’s Claude, using it to translate natural-language and conversational input into structured test cases and detailed steps. To execute tests, we layered computer vision and LangChain‑based orchestration—this duo enables robust UI interaction, verifications, and context-aware navigation through test flows.

Finally we used Bolt.new's stripe capabilities to put in a licensing engine.

In essence, we combined a modern developer experience (Bolt.new), scalable infrastructure (AWS), advanced language understanding (Claude + LangChain), and visual execution intelligence to create a powerful, end-to-end AI testing engine.

Noteable Features

✨ AI Test Creation – Whether you drop in a single scenario or a multi-step user story, our AI breaks it down into actionable tests and streamlined steps with accuracy and speed. Rather than having to write a test to industry standards you can provide an idea of a test, code base or documentation for your application and the AI will extract all relevant tests to cover the application, structure the step definitions in industry standard terminology and deploy them as a batch that you can run with a click of a button in the cloud. Bolt Integration 🚀 Advanced test execution with AI – We leverage the latest and greatest multimodal support for AI to execute the tests this means that we don't use likes of XPath or exact web page routes to run tests meaning our solution is vastly more powerful and less brittle compared to other solutions. Telling you only when there is an issue. You also can see a full log of the entire test run with screenshots, AI's decisions and step completions in realtime. Along with realtime screen and recordings on completion for easy triaging.

🔄 Scaling Execution – QARun.ai spins up on-demand cloud infrastructure in seconds on AWS to execute the tests delivering full test runs in the cloud—no dev machine required. This also means that you can execute as many tests in parallel as you want. Imaging running full regression packs in less than 10 minutes? Bolt Integration 🩺 Test Doctor (Self-Healing) – One of the biggest challenges with testing is false failures and updating the tests and we have built our own test doctor healing functionality. Our intelligent “Test Doctor” continually improves tests as they run, adapting to minor UI changes so scripts stay resilient and maintenance-free. If flags tests where something has changed but the test was still completed suggesting how to update the test. On failures you get single click suggestions of how to restructure the test to resolve the issue.

🚀 Test Learning & Optimization – Our AI learns your test and get smarter and faster over time—AI learns execution flows and optimizes journeys, reducing runtime and improving stability.

Bolt Integration 📌 Bolt.new integration – we used bolt.new to build the application but we also extended Bolt.new to integrate the test workflow. We took the bolt.new github and added in a button that took the code base of the solution the user was working on and ran that through our test generation AI. This meant that as you build a application in Bolt.new you can generate test in QARun.ai to validate and test the solution as you go giving you end to end testing on what you are building and integrating.

📌 JIRA Integration – Pull user stories directly from JIRA into QARun.AI, instantly converting them into tests—no copying or context switching needed. This makes onboarding into the solution quick and easy.

💳 Payment Integration – To support commercialization, we added seamless Stripe integration—administering subscriptions and payments with minimal friction or friction.

Challenges we ran into

  • Optimising start times - getting the test runs to start reliably and quickly was a challenge. We actually found that bolt.new could suggest the enhancements to get the dockerfile for the runner agent's optimised. It actually helped reduce image files by 40% leading to substantial performance improvements.
  • Test Doctor and self healing - there are numerous reasons that cause test failures and getting a self healing system that can work if a page has been updated but to correctly say if a test has failed because the application is broken was a challenge to get running.

Accomplishments that we're proud of

Seamless Bolt.New Integration: We’re especially proud of integrating directly with Bolt.New—developing with it, and thanks to Bolt.New’s open GitHub codebase, embedding a “Test” button right inside the tool. What started as “Wouldn’t it be cool if we could test from Bolt.New?” quickly became a robust, rapid development-to-testing workflow that actually works.

AI That Tests Like a Human—And Learns: We turned the bold idea of “What if AI could see the app and test it like a manual tester?” into reality. Our AI doesn’t just execute steps—it visually analyzes the app, interacts as a real tester would, and goes even further by interpreting test logs, suggesting improvements, and even fixing broken tests automatically.

What we learned

  • The power of vibe coding on Bolt.new - we were able to build out the user journeys and screens rapidly and quickly. Adding new features like batching in about an hour. This allowed us to focus on innovating rather than building.
  • Ability to use Bolt.new to build rapid prototypes for approaches to integrations so we were able to move rapdily. We prototyped the backend in new bolt instances to check best approach of APIs and authentication.
  • For test creation supporting the optimum user flow require a lot of fine tuning of the AI to get the exact structures to parse any test structure.
  • Self-healing testing is crucial—developers lose faith in tools when pipelines break frequently. We wanted to make this as easy as possible and there are lots of reasons for failures which we mapped out.
  • Low-latency cloud execution is a must—feedback loops make or break team trust
  • Integrated workflows such as our bolt.new connection make adoption smooth and frictionless

What's next for QARun.ai - AI testing

QARun.ai is already deployed to hyperscaler and we are looking to move it out of beta into a full production solution to make everyone's testing journey easier.

Built With

Share this project:

Updates

Submission history