QA Sentinel

• 

  QA Sentinel — autonomous release readiness for small software teams.

• 

  Evidence-backed mission report with SHIP, REVIEW, or HOLD guidance.

• 

  Architecture: deterministic QA tools, Strands agent judgment, and human checkpoints.

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for QA Sentinel

Inspiration

Small software teams often ship without a dedicated QA engineer. When a QA professional is available, too much of their time is spent reaching the release candidate, repeating basic checks, collecting evidence, and chasing the same broken links or accessibility gaps. The difficult part is not running every check. It is deciding what actually changes the release decision.

QA Sentinel was built to clear that surrounding work away. It works quietly, gathers objective evidence, and brings a human in only when judgment is needed.

What it does

A release owner gives QA Sentinel a web application URL and a plain-language release goal. A background mission then:

1. validates and reaches the target;
2. inspects the live response, headers, and document structure;
3. runs bounded reliability, navigation, accessibility, security-header, and performance checks;
4. preserves the observation behind every finding;
5. asks a Strands agent to prioritize the tool evidence and produce a structured SHIP, REVIEW, or HOLD decision; and
6. returns a concise report, surfacing only the findings that require a human decision.

The interface shows the mission as it progresses, the checks performed, the evidence collected, and how much routine attention was saved. QA Sentinel never modifies the inspected application.

How we built it

QA Sentinel uses the Strands Agents SDK in Python. The agent receives two custom tools: Page inspection, which returns objective target metadata, and Release checks, which performs deterministic checks and returns evidence-backed findings with stable IDs.

The Strands agent is instructed to use those tools instead of guessing. It receives their observations, prioritizes actual findings by user impact, and returns a Pydantic-validated decision object. Amazon Bedrock is the model provider. The web dashboard and background mission service are intentionally lightweight so judges can run the complete project locally from the public repository.

A clearly labelled evidence-demo mode runs the same deterministic inspection pipeline without model credentials. This makes the complete product experience testable while keeping the Strands mode and its credential state transparent.

Challenges we faced

The central challenge was keeping the product genuinely agentic without letting the model invent QA results. We separated observation from judgment: deterministic tools collect reproducible evidence; Strands decides how to prioritize it and when to interrupt a person.

The second challenge was making autonomy feel trustworthy. Instead of hiding work behind a spinner, the dashboard exposes a compact activity trail and gives every issue an ID, severity, observed evidence, and concrete next action.

Accomplishments that we're proud of

• A complete URL-to-release-decision workflow, not a chat-only prototype.
• Custom Strands tools grounded in bounded, read-only inspections.
• Structured agent output that the application can validate and render reliably.
• Human checkpoints tied to material release risk.
• Safety controls for protocols, private-network targets, response size, and same-origin link checks.
• A polished interface that demonstrates the product end to end.

What we learned

The most useful professional agent does not replace expert judgment. It removes the repetitive work around that judgment and arrives with enough evidence for the expert to decide quickly. Strands made the tool-use loop small enough to keep the orchestration understandable while still supporting structured output and production observability.

What's next for QA Sentinel

• Browser-level journey execution with screenshots and console evidence.
• Scheduled regression missions and release-diff comparisons.
• GitHub issue drafting behind a human approval checkpoint.
• Deployment to Amazon Bedrock AgentCore.
• Strands Evals coverage for tool selection, groundedness, and decision quality.

Built With

• amazon-bedrock
• strands-agents