Cloud Janitor

Dashboard
Findings, Audit Trail, Remediation vs Rollback, Agent Reasoning
Remediation Explainer
Policy Suggester

Inspiration

Every AWS account accumulates the same wreckage: an idle ElastiCache cluster nobody remembers spinning up, a security group left wide open "just for testing," an EBS volume billing away in silence. The tools built to catch this — Cost Explorer, Security Hub, Cloud Custodian — are good at finding the problem and bad at the part that actually matters: deciding what's safe to touch.

Cloud Custodian is the clearest example. It's mature, CNCF-backed, and still runs on static YAML policies a human has to write and maintain by hand. It doesn't reason about a resource's dependencies, and it doesn't explain a remediation in plain English before running it. Finding that project mid-build sharpened our pitch rather than derailing it: the industry doesn't need another scanner. It needs something a human will actually trust to pull the trigger.

That's the gap Cloud Janitor is built for — and it's why we spent our spec time on the approval gate and the explainability layer, not just the scanners.

What it does

Cloud Janitor is a multi-agent system that audits an AWS account for cost waste and security exposure, reasons about what it finds, and proposes a fix — but never applies one without a human explicitly approving it.

FinOps Auditor — finds idle and oversized resources burning money (idle ElastiCache, unattached EBS, idle EC2)
SecOps Guard — finds exposed, misconfigured, or non-compliant resources (open Redis/MySQL/Postgres/Mongo ports, SSH open to the world, unencrypted storage)
Remediation Architect — turns findings into a real Terraform HCL plan, with a rollback plan generated before anything executes, and blocks any resource with live dependents
A second wave of AI agents layered on top — natural-language query interpreter, explainer, policy suggester, anomaly detector, drift detector, resource tagger, incident policy generator, and multi-account orchestrator — all live in the dashboard, routed through a configurable LLM endpoint (BYO-endpoint supported) with graceful fallback if the call fails, and a hard JANITOR_AI_ENABLED=false kill switch for zero network egress
Approval Gate — a typed state machine, not a dialog box: APPROVE <resource-id> only, case-sensitive, three failed attempts locks it, rollback is a deliberate two-step confirmation It ships three ways: a pip install cloud-janitor CLI (scan, approve, rollback, dashboard, mcp), an optional Streamlit dashboard, and an MCP server exposing 10 tools that any MCP-compatible client — Kiro, Claude, Cursor — can call directly.

The demo scenario, the "Ghost Cluster," is deliberately mundane: an idle ElastiCache instance behind an exposed security group, with a real dependency link between them so the blocking logic has something to demonstrate. That's the point — this is the boring, everyday waste sitting in every real AWS account, not a contrived edge case.

How we built it

We treated Kiro's spec-driven workflow as non-negotiable, not a formality. Every feature started as a committed spec in .kiro/specs/ — requirements, design, and a task breakdown — reconciled before Kiro generated a single implementation task. Six specs shipped this way (cloud-janitor, cloud-janitor-phase-bc, provider-agnostic-backend, savings-tracker-localstack, audit-remediation, production-readiness), totaling 232 tracked tasks, 223 complete. A .kiro agent hook (update-docs-on-stop.kiro.hook, triggered on agent stop) automatically reconciles README structure against the session's actual diff — small, but it's real automation wired to a real trigger, running unattended across the whole build. SPEC_COMPLIANCE.md is auto-regenerated on every commit via a git post-commit hook, so "done" can't quietly drift from what's actually implemented.

Kiro did the implementation, steered by .kiro/steering/ documents written in prohibition-first language rather than soft preference. We found Kiro follows a hard "never do X, do Y instead" instruction far more reliably than a polite suggestion, which mattered most for enforcing uv over pip across 500+ commits. We paired Kiro's implementation work with a manual architectural review pass: spec reconciliation, conflict-checking, and eventually a full security self-audit against the codebase, which is where the story below picks up.

The backbone is a Python src-layout package driven by an Orchestrator that owns the agent pipeline, the approval state machine, and every subprocess call out to Terraform. Findings pass between agents through a shared, schema-versioned findings_store.json. Terraform runs against LocalStack via tflocal, seeded with the fixed Ghost Cluster scenario so the whole pipeline — waste detection, security findings, a blocked remediation, and a live apply/rollback cycle — is reproducible without a real AWS account. We also built our own MCP server from scratch (aws_janitor_mcp.py, FastMCP) rather than only consuming one — 4 core infrastructure tools and 6 AI-reasoning tools, all callable over stdio by any MCP client.

Challenges we ran into

We audited ourselves, and it wasn't flattering. Late in the build we had Kiro produce a structured findings report (.kiro/cloud-janitor-findings.md) against our own main branch, tagged by evidence level ([EXECUTED], [CONFIRMED], [STATIC]). It surfaced real, serious problems:

Every subprocess inherited the full parent environment, meaning terraform apply (arbitrary downloaded provider code) had access to our OpenRouter key and AWS secrets. Fixed with _build_subprocess_env(kind) — an explicit allowlist built per child-process type (terraform vs. hook), so neither API keys nor AWS credentials are ever handed to Terraform provider code that doesn't need them.
Raw Terraform error output with account IDs, ARNs, and VPC IDs was landing unscrubbed on screen and in the audit log. Fixed with a _redact() layer applied before anything touches a log or the UI — strips AWS account IDs, ARNs, AKIA*/ASIA* access keys, sk-or-* API keys, and VPC/subnet IDs.
A path mismatch: the findings store was resolving relative to the project root instead of output/, risking silent writes to the wrong place. Fixed by centralizing path resolution.
A false-failure bug: a clean, healthy AWS account (zero SecOps findings) was reported as a pipeline failure, because the validator required both finops and secops findings to be present. Fixed by tracking agents_completed explicitly, separate from findings count.
LLM egress with no controls. Every agent sent resource metadata to OpenRouter with no guarantee where it landed. Fixed with a BYO-endpoint option, a hard JANITOR_AI_ENABLED=false kill switch, and a documented "Security Hardening" section in the README. We shipped the fix as a real release, not just a README edit — the work landed as v0.2.0 on PyPI, not a silent patch to the same version number.

Implementation completeness isn't integration completeness. We built a full second wave of agents — natural-language querying, anomaly detection, drift detection, policy suggestion — fully tested and working in isolation, and for most of the build, not fully wired into the dashboard. Getting powerful agents built is the easy 80%. Surfacing them somewhere a judge can click through in three minutes is the harder 20% — and it's exactly where we spent our last hours: the standard audit flow now surfaces anomaly and drift results directly, not just the natural-language query path.

Src-layout migration under deadline pressure. We restructured the whole package from a flat layout into a proper installable src/cloud_janitor/ package so it could ship to PyPI with a real CLI entry point, without breaking any of the tests that already existed against the old import paths.

Our own test suite found a real bug. We hold tests to a hostile-reviewer standard — no pass-by-default assertions, property-based testing throughout with Hypothesis. Fuzzing surfaced an edge case in the policy suggester's exclusion filter matching on a partial key instead of the full composite key — exactly the kind of bug hand-picked example tests sail past.

Getting Terraform, tflocal, and LocalStack working from Windows/PowerShell + WSL was its own war of attrition. Path resolution that worked in WSL broke in Git Bash, and vice versa. We eventually replaced the bash seed script with a portable Python/boto3 implementation to remove the shell dependency entirely.

Accomplishments that we're proud of

1,295 passing tests, zero pass-by-default assertions, on a system that touches real infrastructure decisions
Shipped to PyPI (pip install cloud-janitor) with a real CLI, not just a script you clone and hope works — currently at v0.2.0
Went from a self-identified CRITICAL-severity finding (secrets leaking to every subprocess) to a documented, tested Security Hardening section in the README, shipped as a versioned release, in the same working session
500+ commits since June 26, spec-first the entire way — .kiro/specs/ shows the spec guiding the build throughout, not written once and abandoned
A demo that survives a judge's machine not having Terraform installed — TF_CMD=echo and a bin/tflocal wrapper mean the full approve → execute → rollback flow runs cleanly in dry-run mode
An approval gate that isn't cosmetic: a typed state machine with a real lockout invariant, verified by property tests
A complete AWS provider backend — not fixtures pretending to be AWS — with GCP/Azure left honestly labeled as interface stubs rather than faked
A self-built MCP server (10 tools) rather than only consuming one
All eight AI agents fully live in the dashboard — NL query, explainer, policy suggester, anomaly detector, drift detector, resource tagger, incident policy generator, and multi-account orchestrator — closed out in the final hours before submission ## What we learned

The scanning problem is basically solved industry-wide. Cost Explorer and Security Hub already tell you what's wrong. What nobody has built well is the trust layer: the moment a tool says "here's the exact change, here's how to undo it, are you sure" and means it. That has to be the thing the architecture is organized around from day one, which is why the approval gate is a first-class state machine here, not a UI afterthought.

We also learned that verifying an AI coding agent's claims against literal file contents and real terminal output — every time, not just trusting a summary — is what actually catches bugs like the ones above. Steering an AI coding agent is its own discipline, separate from writing good specs: soft preferences get ignored under deadline pressure, explicit prohibitions with substitution tables get followed. And "human approval before anything executes" is a harder promise to keep at the resource level than the file level. Approving one resource's remediation could previously have silently applied everything else sitting in the same generated Terraform file — that's now fixed with per-resource apply isolation, staging only the approved resource's HCL into an isolated working directory before apply.

What's next for Cloud Janitor

Session/tenant scoping for the findings store and rollback artifacts, so a shared deployment can't bleed one account's data into another's dashboard
A terraform import step (or an explicit statement of scope) so remediation against real, pre-existing AWS infrastructure behaves the way the docs promise, not just against LocalStack's demo state
Pseudonymizing resource names before they ever reach an LLM, as a technical guarantee rather than a policy promise
Open-core distribution: free scanner/CLI/MCP server, paid tier for team audit history, scheduled scans, and multi-account scale
A hosted MCP endpoint — point Kiro, Claude, or Cursor at a URL and get the tools with zero install
Multi-cloud (GCP, Azure) once the AWS backend has more real-world mileage on it