QualiAgent

I built QualiAgent to help teams understand what a code change could break, what to test first, and when a release still needs human review.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for QualiAgent

QualiAgent analyzes a unified code diff and produces:

concise QA decision brief deterministic overall risk score and risk level Affected system areas Evidence-linked risk findings Prioritized regression tests Expected outcomes for each test release recommendation that explicitly preserves human approval

The application includes two prepared examples:

  • web authentication change involving email normalization and password validation Solana transaction-flow change involving fallback submission, confirmation behavior, timeouts, and asynchronous polling

Users can also paste their own unified diff for analysis.

How I built it

QualiAgent is built with Next.js, React, TypeScript, Tailwind CSS, and the OpenAI Responses API.

The frontend provides the change-input workspace and presents the analysis as a structured QA report. A server-side API route sends the diff to the OpenAI model and validates the structured response before returning it to the interface.

Risk labels are derived deterministically from the validated numerical score:

  • 0–39: Low
  • 40–69: Medium
  • 70–100: High

This keeps the overall risk label consistent and avoids relying on multiple independent sources of truth.

The application is deployed on Vercel and connected to a public GitHub repository.

Challenges

One important challenge was balancing useful AI reasoning with predictable application behavior. The analysis needed to remain detailed and context-aware while still following a stable structure that the UI could reliably validate and display.

Another challenge involved deployment execution time. The server-side analysis initially exceeded the configured limit in production. After reviewing Vercel runtime logs, I increased the allowed execution time and redeployed the application. The Web and Solana production tests then completed successfully with HTTP 200 responses.

I also separated individual finding severities from the overall risk level. Individual risks remain independently assessed, while the overall label is calculated from one validated numerical score.

What I learned

This project reinforced that AI-assisted quality engineering should support human judgment rather than replace it.

A useful QA agent should not only identify risks. It should explain the evidence, prioritize the next actions, communicate uncertainty, and clearly show where qualified human review is required.

I also gained practical experience with structured AI outputs, runtime validation, deterministic risk classification, server-side API integration, production deployment, and end-to-end testing.

What's next

The next stage would connect QualiAgent directly to pull requests and CI/CD pipelines.

Planned improvements include:

GitHub pull-request integration Automatic diff collection Repository-aware analysis Historical risk comparison Test-management integrations Team review and approval workflows Exportable QA reports Custom risk policies for different products and industries

The long-term goal is to make QualiAgent a practical release-intelligence layer for engineering and QA teams.

Built With

Share this project:

Updates