MorningGuard

AI-assisted smoke testing that turns deterministic failures into evidence-grounded diagnoses and actionable next steps.

Comment

Inspiration

In many cases, system problems are discovered only after the working day has already begun. Users cannot log in, applications are unavailable, and engineers can only investigate after a support ticket is raised. Then they must check logs, responses, screenshots, and database state to find the root cause and fix the problem.

MorningGuard was created to move this process earlier and make failure analysis faster, clearer, and more actionable.

My broader goal is to explore how Generative AI can support software quality in a safe, explainable, and human-reviewed way.

What it does

MorningGuard is a deployed proof-of-concept smoke testing system that combines deterministic Web, API, and database checks with AI-assisted failure diagnosis.

It runs real checks against real systems and records deterministic PASS, FAIL, or ERROR outcomes.

For failed checks, MorningGuard collects supporting evidence such as error messages, API responses, database results, and screenshots. Only those failed checks are passed to an AI Diagnosis Agent.

The agent returns a structured diagnosis containing:

  • what happened
  • the likely cause
  • diagnosis confidence
  • supporting evidence
  • recommended next action

The AI does not change or override the original deterministic result.

How it works

The V1 flow is:

Deterministic Web/API/DB checks
→ evidence collection
→ FAIL-only AI Diagnosis Agent
→ OpenAI Responses API structured output
→ local validation
→ persistence
→ dashboard

The OpenAI model never writes directly to the database.

MorningGuard validates the structured AI response locally before storing it. This keeps deterministic execution and AI reasoning clearly separated.

How I built it

MorningGuard was built with:

  • Python 3.12
  • FastAPI
  • Playwright
  • PostgreSQL
  • SQLAlchemy
  • Alembic
  • Pydantic
  • Docker
  • Railway
  • OpenAI Responses API
  • GPT-5.6 Sol
  • Pytest

Codex was used as an active development partner for implementation, targeted code changes, debugging, automated tests, and deployment preparation.

ChatGPT was used for architecture review, LLM-as-a-Judge checks, prompt review, documentation, and visual design.

Development was done iteratively in small steps, with automated tests and manual review after important changes.

Challenges

One of the main challenges was keeping deterministic test execution and AI reasoning clearly separated.

The AI had to provide useful explanations without being allowed to change PASS, FAIL, or ERROR outcomes.

Another important challenge was evidence grounding. AI output is useful only if it can be connected back to real execution evidence.

For this reason, MorningGuard validates evidence references and structured model output before persistence.

A further challenge was building a real deployed end-to-end workflow within the hackathon timeframe instead of presenting simulated or precomputed results.

Accomplishments that I'm proud of

MorningGuard became a real, deployed end-to-end proof-of-concept system within the Build Week timeframe — not a simulated or precomputed demo.

The final automated test suite contains:

170 passed

A verified deployed Railway run executed:

  • 9 checks
  • 6 PASS
  • 3 FAIL
  • 0 ERROR
  • 3 AI-assisted diagnoses

The complete public workflow was verified from the browser through deterministic execution, evidence collection, AI diagnosis, local validation, persistence, and final dashboard rendering.

What I learned

The project reinforced that AI can add real value on top of deterministic testing when it is constrained by clear rules.

The most important lessons were:

  • deterministic checks should remain the source of truth
  • AI should support investigation, not replace deterministic decisions
  • structured output makes AI responses easier to validate and integrate
  • evidence grounding is essential for trustworthy diagnosis
  • local validation should happen before persistence
  • human oversight remains important in quality-critical workflows

What's next

MorningGuard v1 is a deployed proof of concept.

The next step is to evolve it into a true multi-agent architecture with:

  • an Agent Orchestrator
  • specialized AI agents
  • risk-based smoke planning
  • cross-system correlation
  • stronger multi-system support
  • strict human-in-the-loop review and control

The goal is to keep the deterministic safety foundation of v1 while adding more intelligent planning, investigation, and coordination capabilities.

Built With

Share this project:

Updates