Inspiration
After a cloud incident, the hard part is not usually one missing log line. It is turning scattered evidence into a report that a security lead can trust quickly enough to act.
I built TraceGuard around a realistic GCP cleanup scenario: a Cloud Run service exposed to allUsers, a temporary admin with roles/owner, suspicious IAM/token activity, broad Terraform ingress, and disabled repo protections. Any one of those signals can be noisy. Together, they need evidence-backed triage, not a confident AI summary with no receipts.
What It Does
TraceGuard takes GCP audit logs, IAM policy JSON, Terraform snippets, alert text, and repo metadata, then turns them into a grounded incident report. It identifies confirmed findings, confidence, severity, impact, remediation, detection logic, CWE references, and MITRE ATT&CK mappings.
The important rule is simple: a confirmed claim must cite source evidence IDs. If evidence is weak or missing, TraceGuard reports the result as inconclusive instead of inventing certainty.
How I Built It
I built the local agent in Python with a lightweight HTML/CSS/JavaScript web UI. The hosted version runs on Google Cloud Run and uses Gemini on Vertex AI for narrative synthesis after deterministic findings are produced.
For the Arize track, I instrumented the run path with Phoenix/OpenTelemetry, added a pinned Phoenix MCP runtime path for read-only introspection, and built code evals for grounding, claim hygiene, remediation quality, detection usefulness, severity calibration, and duplicate pressure.
The app shows an Observe -> Evaluate -> Improve loop: Phoenix/OTEL and MCP show runtime observability, evals measure output quality, and the improvement planner proposes the next-run change from eval and MCP receipts.
Challenges
The hardest part was keeping the AI useful without letting it become the source of truth. Gemini can summarize the deterministic findings, but it is not allowed to invent security facts. I added validation so Gemini-generated briefs only pass when evidence references match known parsed evidence IDs.
Another challenge was turning Arize from passive observability into part of the agent loop. I did not want Phoenix traces and evals to be just a screenshot after the run, so I wired them into the workflow: TraceGuard records the run, checks grounding quality, reads MCP status, and uses those receipts to produce a concrete next-run improvement plan.
Accomplishments
I’m proud that TraceGuard has a working hosted Cloud Run app, a public repo, a live proof endpoint, Gemini 3 validation, Phoenix/OpenTelemetry tracing, Phoenix MCP status, quality evals, and a visible improvement loop.
The sample run produces 11 findings, 8 critical/high findings, 0 unsupported confirmed claims, and about a 94% eval average, with evidence IDs carried through the final report.
What I Learned
I learned that agentic security tooling needs strict evidence boundaries. The fastest way to make a security copilot untrustworthy is to let it sound certain before the data supports it.
I also learned that observability is more valuable when it feeds back into the agent workflow. Traces and evals should not just decorate a demo; they should help explain what changed and what the next improvement should be.
What’s Next
Next I would connect TraceGuard to real GCP asset inventory, Security Command Center findings, Cloud Logging exports, and repository APIs. I would also expand the Phoenix MCP loop so the agent can compare historical runs, identify recurring weak evals, and recommend more targeted detector or report improvements over time.
For production, I would put the hosted app behind IAM, IAP, or Cloud Armor instead of a shared access key.
Log in or sign up for Devpost to join the conversation.