IncidentDocket

Give Codex only the Windows evidence it needs. Get privacy-bounded, evidence-linked incident reports without unrestricted diagnostic access or unsupported root-cause claims.

Comment

Inspiration

Windows incident investigation often begins with only a timestamp and a vague symptom. The usual next step is to gather more data, but broad diagnostic dumps can expose usernames, file paths, identifiers, network details, credentials, and unrelated system activity. Giving an AI agent unrestricted access makes that privacy problem larger; collecting too little leaves developers with nothing they can confidently investigate.

Sessions on NeMoClaw and real-world agent systems at Build-a-Claw Tokyo sharpened a concern I had already been considering: as agents gain access to operating-system tools and sensitive data, guardrail design becomes as important as model capability.

I wanted to place a controlled, inspectable evidence layer between Windows and the reasoning model—one that gives the model enough context to help without treating the whole machine as its context window.

IncidentDocket also became a practical test of a more personal question: how far could a first-time hackathon participant take a real developer tool when Codex was used as an implementation and review partner, rather than as a one-shot code generator, while product judgment remained human-owned?

What it does

IncidentDocket is a privacy-bounded Windows evidence collector for developers and first-line technical support who maintain Windows applications or drivers. It turns an incident timestamp and a vague symptom into a narrow, evidence-linked Markdown support report without giving the reasoning model unrestricted diagnostic access.

It runs locally as a stdio MCP server and exposes four tools:

plan_collection → collect_incident_window → inspect_evidence → export_support_report

The workflow:

  1. validates an offset-bearing incident timestamp, a bounded time window, and a small allowlist of evidence sources;
  2. collects only the requested synthetic or Windows evidence;
  3. masks known sensitive identifiers before processed evidence is saved or returned;
  4. assigns stable evidence IDs so the client can inspect only what it needs; and
  5. validates every cited ID and the shape of the model-assisted analysis before exporting a report.

Live collection is limited to Windows 11, runs as the current user without automatic elevation, and supports four sources: time-bounded System and Application Event Log entries, an OS snapshot, and a display-driver snapshot. A deterministic gpu-driver-reset fixture demonstrates the same complete MCP workflow on Windows, macOS, or Linux without reading real machine evidence.

IncidentDocket keeps observations separate from conclusions. A report can contain up to three hypotheses or explicitly state that the evidence is insufficient. Hypotheses are limited to low or medium confidence, must cite evidence that actually exists, and must state what is not proven. Collection-time snapshots cannot be the sole support for a hypothesis, and every report warns that temporal proximity does not prove causation.

The tool does not repair the machine, execute recommendations, or autonomously declare a root cause. It never saves or returns raw collector evidence, and every exported report still requires human privacy review because masking is defense in depth, not complete anonymization.

How we built it

IncidentDocket is an ESM TypeScript application with Windows PowerShell 5.1 collectors. It uses the Model Context Protocol SDK and Zod as its only runtime dependencies; hashing, UUIDs, paths, process execution, storage, and Markdown rendering use Node.js standard libraries. The server itself implements no network client or direct OpenAI API integration.

The project was built through a human-controlled, phase-gated process, not a single broad prompt. I defined the problem, audience, threat model, product scope, privacy and causal boundaries, acceptance gates, and release decisions. DESIGN.md became the sole product specification, while Codex and GPT-5.6 operated inside that contract.

The implementation sequence was deliberate:

  1. lock the product contract and explicit exclusions;
  2. build strict schemas, stable ordering, evidence IDs, and the deterministic fixture;
  3. add masking, storage controls, Markdown safety, and regression tests;
  4. connect the complete four-tool MCP workflow to bounded model reasoning;
  5. add the four Windows collectors only after the fixture path was stable; and
  6. verify the packed CLI, packed MCP server, installer, checksums, CI, and public Release.

GPT-5.6 Luna handled bounded implementation and remediation tasks. GPT-5.6 Sol was used in separate sessions for deeper architecture, privacy, platform-boundary, supply-chain, and release-readiness reviews. I triaged the findings, rejected scope expansion, chose the required fixes, and kept the decision to publish human-controlled.

Those reviews changed the shipped software. They found, among other issues, incomplete revalidation of caller-supplied plans, a Windows 11 check that could admit Windows Server, gaps in installer checksum boundaries, and release paths that did not fail closed. Each accepted behavior change received a focused regression test and passed the same build, package, audit, and acceptance gates.

The detailed planning and audit sessions were conducted mainly in Japanese, my first language, so I could reason precisely about the technical and privacy trade-offs. Codex also helped turn that work into the English documentation and demo material presented to reviewers.

Challenges we ran into

  • Separating evidence from interpretation. An event near an incident is relevant context, not proof of root cause. We encoded that distinction in evidence IDs, confidence limits, required not_proven statements, snapshot rules, and an explicit insufficient-evidence outcome.

  • Treating diagnostic data as untrusted. Logs can contain secrets, malformed markup, or text that resembles instructions to an agent. IncidentDocket projects only allowlisted fields, masks recognized sensitive values, drops evidence that still matches recognized sensitive patterns after sanitization, and never executes evidence content. The remaining data is still labeled untrusted and requires human review.

  • Making the complete product safe to evaluate. A judge should not need to expose a real machine or rebuild the repository. The synthetic fixture exercises the actual four-tool workflow. The documented Windows installation path verifies the setup ZIP before extraction, then verifies the bundled npm package, the installed fixture, and Codex MCP registration.

  • Keeping the scope reviewable. Early plans considered more evidence sources, HTML reports, and a broader audience. I reduced the submission to four sources, Markdown output, one fixture, and four MCP tools. That smaller surface made the privacy boundary testable and the product coherent within the hackathon.

Accomplishments that we're proud of

  • A complete, runnable path from bounded collection planning to an evidence-linked support report—not just a mock UI or a predetermined demo result.
  • A synthetic fixture that lets judges exercise the full model-assisted workflow without exposing real diagnostic data.
  • Privacy and causal boundaries enforced in schemas and code rather than left only to prompting.
  • Stable evidence identities, selective inspection, explicit uncertainty, and a first-class insufficient-evidence result.
  • Automated regression coverage on Windows and Ubuntu, plus packed CLI/MCP, current-working-directory isolation, installer-acceptance, and release-workflow checks.
  • A public MIT-licensed repository, checksum-verified Windows installer, reproducible Release assets, and fail-closed publication controls.
  • As a first-time hackathon participant, shipping my first public developer tool with an installer, CI, release pipeline, English documentation, and a complete demo video.

What we learned

The biggest lesson was that effective AI-assisted development is not the same as accepting generated code. Codex became more useful as the constraints became more explicit: a single source of product truth, narrow work units, phase gates, negative tests, exact release checks, and separate adversarial-review sessions.

I also learned that the strongest agent boundaries live outside the model. Schemas, allowlists, time limits, masking, storage restrictions, evidence identity, confidence limits, and report validation are inspectable controls even when model output varies.

Repeated review mattered as much as initial implementation. Some of the most consequential improvements came from asking a fresh model session to challenge assumptions that earlier implementation and review passes had accepted.

Most importantly, this project showed me how much Codex can expand an individual builder's reach. It helped me implement, test, audit, document, package, and release something I had not shipped before. Human judgment was still required to define the problem, decide what not to build, evaluate findings, and determine what was ready to share.

What's next for IncidentDocket

The immediate next step is validation beyond the synthetic fixture: privacy-reviewed use on clean Windows environments, clearer source-specific warnings, and feedback from developers and support engineers investigating real applications. That work must preserve the rule that real machine evidence never enters fixtures, screenshots, or the repository.

Release hardening can continue with signed checksums and stronger provenance, scheduled clean-environment installation tests, richer Markdown report templates, and optional issue-tracker integrations. Any new evidence source would remain explicitly allowlisted and would need its own privacy and failure-mode review.

Longer term, IncidentDocket could develop from this Windows implementation into an evidence firewall for AI agents: a portable contract for giving agents least-privilege access to sensitive technical data while preserving provenance, bounded evidence budgets, masking, trust labels, evidence-linked claims, and explicit uncertainty.

That is a direction, not a claim about the current product. The central boundary would remain unchanged: collection stays separate from inference, raw evidence stays untrusted, human review remains required, and automated repair stays out of scope.

Built With

Share this project:

Updates