-
-
Recorded Kiro CLI: 120 tests passed, then five successful source edits. Earlier test evidence is stale for those edits.
-
Judge Fast Path: inspect the real saved Kiro case without signing in.
-
Opt-in PR Proof Chain: observed GitHub CI at the exact SHA, not required-check certification.
-
Browser-local Live Investigator: analyze a selected Kiro messages.jsonl without server upload.
-
Filmed M19 commit: two successful GitHub Actions jobs; provider evidence, not proof of correctness.
Specter Audit — Trust the Evidence
The tests passed. Then the agent edited the code. What does the evidence actually prove?
JUDGE DEMO — M305 reviewed and browser-tested build (main 4bb872e9), no signup: https://specter-audit-judge-m305-nvhoa1691993-6852s-projects.vercel.app/
Official V6H judge video (2:23, filmed before the post-video fixes; public): https://youtu.be/yabjJ5JpsS0
Public GitHub, MIT license, root .kiro/: https://github.com/ShenJun93/specter-audit
See the problem in one click
An AI coding agent reports 120 passing tests. It then makes five more successful source edits. Those earlier results are real recorded observations—but they do not establish the final code was tested. Likewise, a green GitHub Actions badge may refer to an ancestor commit instead of the code now under review.
Specter Audit is a read-only evidence investigator for developers and maintainers reviewing AI-generated work. It relates agent claims to test receipts, source edits and commit SHA identities, then distinguishes what is supported from what remains unknown.
Try the judge path
- Click JUDGE FAST PATH → INVESTIGATE THE MIDNIGHT REWRITE.
- Wait until authentic Kiro session Case 04 loads; click REVEAL FINDING NOW (or EXPOSE THE ALIBI). Expect STALE LOCAL TEST EVIDENCE: 120 reported PASS, five later edits, no subsequent passing retest observed in that saved session. This is not proof the code broke or that no tests ran elsewhere.
- Open Specter Arena from the homepage. Compare provider-reported CI with a reviewer-controlled, one-behavior challenge: Base FAIL / Head PASS. The result is a bounded self-reported demonstration, not independent test attestation, merge approval or a correctness certificate.
- Try the missing-evidence case: INSUFFICIENT EVIDENCE is a valid UNKNOWN outcome. The app will not invent passing tests.
Why this matters
- Decision-focused: tells the reviewer what to do next—rerun tests after the final edit, check the exact commit, or request focused verification.
- Temporal and SHA-aware: separately tracks which code state the test receipt covered and which commit the provider CI observed.
- Evidence-provenance labels: genuine but editable Kiro logs, provider observations and owner-controlled fixture results are never silently promoted to independently authenticated proof.
- Browser-local Kiro analysis, tested: a focused Edge test used synthetic messages.jsonl files for stale, current, malformed and over-limit inputs. It observed no post-selection network requests, no WebSocket traffic and no exposure of a private-path canary. This is a bounded browser test, not a formal security certification.
- Halloween design with a purpose: agent claims become alibis, chronological records become exhibits and uncertainty is made visible in a detective-style verdict flow, instead of hiding behind green badges.
Kiro was used to build real functionality
Kiro CLI made recorded, successful source/test edits—not just a UI prompt.
- M1: foundational Git-inspection and claim-evaluation implementation. Source commit.
- M5A: GitHub Actions evidence adapter; the saved real Kiro CLI sequence with 120 PASS → five later edits. Source commit.
- M6E: Kiro-authored early
src/kiro-session-staleness.mjsimplementation and tests. Source commit. - M118: Kiro CLI generated the initial Arena case-report source and tests; its noninteractive test run was blocked. ChatGPT/WAG subsequently corrected tests and integrated Docker. Bounded Kiro receipt.
- Specs and steering: committed root
.kiro/specs/and.kiro/steering/document requirements, evidence boundaries and development decisions. Kiro session evidence.
Authorship limits are explicit: saved Kiro CLI logs are editable, local self-records, not signed proof. ChatGPT/WAG and other tools helped develop UI, integration and QA. We do not claim a deployed Kiro hook, Crew, MCP, powers or custom agent without direct evidence.
Technology, reproducibility and scope
Free, open-source, read-only JavaScript/Node.js 24+ app. Current M305 judge-demo source: 4bb872e9. Historical M204 judge-demo source: e68b39d, sharing Git tree 3c5633e... with then-current main 8079e6f. V6H was filmed from the older M130 source: d8f8e7a, not the current M305 judge demo. Post-video changes fix the Case 04 loading race and add optional, user-triggered Arena review-note copying. The new review-note feature was built by ChatGPT/WAG, not Kiro. M204 passed targeted local browser judge checks; these are owner-run QA observations, not independent attestation or correctness/security certification.
The intended audience is AI-assisted solo developers and PR reviewers; no external customer adoption or independent user studies are claimed. Arena reports one owner-controlled behavioral test; the browser does not independently verify its execution.
Historical evidence preserved: M130 — exact V6H filmed demo, V5D original public video and the earlier M75 demo remain available for reproducibility. The historical M204 demo was a post-video code revision, distinct from filmed M130; M305 subsequently added importer trust safeguards.
Trust the evidence, not the claim.
Built With
- git
- github-actions
- javascript
- kiro-cli
- kiro-specs
- kiro-steering
- node.js
- vercel
