Inspiration
I use AI coding tools regularly, but as an independent developer, I do not have an unlimited AI budget. I wanted to understand what I was actually getting from that usage—not just how many prompts or tokens I consumed, but whether a Codex session was useful and whether the result was ever verified.
I also spoke with friends who manage technical teams. They were asking a similar question at a larger scale. Their companies were spending more on AI development tools, but it was still difficult to see what that work had actually produced.
I could already track prompts, token usage, and cost. What I could not easily see was whether the session went well, where I had to step in, or whether the final result had been tested.
Kea does not try to calculate productivity or ROI. It gives you a record of what happened during the session and separates what Codex said it completed from what the recorded evidence can support.
What it does
Kea is a lightweight, project-local companion for Codex CLI.
It records observable activity from a Codex session and turns it into a local session brief. The report helps developers and technical leads see:
- what the session was trying to accomplish;
- what Codex attempted;
- where the developer changed direction;
- what Codex reported as the outcome;
- what the recorded evidence supports;
- what testing or verification was observed;
- and what is still unknown.
Kea does not automatically accept the model's analysis as correct. It checks the findings against the session record before placing them in the final report.
A deterministic validator checks whether each citation exists and whether the cited evidence makes sense for the claim. It also distinguishes between an attempted action and a recorded result.
For example, starting a command does not prove that it finished successfully. Editing a file does not prove that the feature works. If a model says all tests passed, Kea looks for the successful test output rather than accepting the statement on its own.
When a finding is not supported, Kea can reject it, lower its confidence, or revise it into a smaller claim that the evidence does support. The validation audit shows what changed and why.
Raw session recordings stay on the user's machine. Provider-backed analysis requires explicit permission and a user-supplied API key. Kea also removes or masks sensitive information before sending eligible evidence to the model.
How we built it
The project began with a brainstorming session between me and GPT-5.6. We explored several Build Week ideas and narrowed them down to three.
I chose Kea because it addressed a problem I was already having with Codex, and the scope felt realistic for the hackathon. It also gave me a chance to test the product on the same type of sessions I was using to build it.
I made the final decisions about what Kea should do, what it should avoid claiming, how the local workflow should work, and what could realistically fit into the MVP.
I used GPT-5.6 to challenge the idea, identify weak points, define the evidence rules, break the work into milestones, and prepare implementation prompts for Codex.
Codex handled much of the implementation. It built the session-capture hooks, local recorder, evidence-processing pipeline, validator, report generation, tests, fixtures, debugging tools, and credential-free judge demo.
After each milestone, I reviewed what Codex produced, checked the output and tests, and either accepted the result or changed the direction.
Kea uses Codex hooks to capture session activity locally. When Codex emits a Stop event, Kea launches a short-lived worker. The worker waits for the recording to settle, processes the evidence, updates the local report inbox, and exits. This keeps normal usage passive without requiring a permanent background service.
The project also became partly recursive. While Codex was helping build Kea, Kea recorded part of that same development activity. I did not use the report to guide the build in real time. Afterward, I reviewed and sanitized the session and used it as the basis of Kea's reference demo.
Challenges we ran into
One of the hardest parts was deciding what Kea could honestly conclude from incomplete evidence.
A coding session may contain a lot of activity without proving that the intended result worked. Commands can be started without finishing. Files can be edited without the feature being tested. An assistant can confidently report success even when the successful output was never recorded.
That is why Kea separates a reported outcome from a supported outcome. One is what Codex says happened. The other is what the session record can verify.
Citation validation was also difficult. A deterministic validator cannot fully understand every natural-language claim, so I limited it to checks that code can perform reliably. It verifies that the citation exists, that the evidence type fits the claim, and that a claimed result points to an actual result rather than only an attempted action.
Another challenge was keeping the tool passive without running a permanent background service. Kea uses Codex hooks for capture and launches a short-lived worker only after the session stops.
Privacy was a major concern as well. Coding sessions can contain file paths, prompts, command output, Git information, environment details, and secrets. Kea keeps the raw recording local and only sends sanitized evidence after the user gives permission. If a session is too large, Kea stops and tells the user instead of quietly dropping part of the record.
We also had to avoid overbuilding. It was tempting to add hosted dashboards, authentication, team management, cross-session analytics, and support for several coding agents. I decided to prove the capture, evidence, and validation flow first.
Accomplishments that we're proud of
I am proud that Kea is more than a mock interface. The credential-free demo runs the real evidence builder, schema parser, validator, persistence layer, report renderer, disposition logic, and static report inbox.
The reference demo intentionally includes three flawed candidate findings:
- one is rejected because it cites evidence that does not exist;
- one is downgraded because the claim is stronger than the evidence;
- and one is amended because it is missing a required citation.
This lets judges see the validator challenge the generated analysis instead of simply displaying whatever the model returned.
I am also proud that Kea is allowed to say unknown. If the session does not contain enough evidence, the report does not invent a confident answer just to look complete.
Another meaningful part of the project was using a real, sanitized Kea development session as the reference demo. Codex helped build Kea, Kea captured part of that work, and the resulting session became a realistic test of the product.
The demo also requires no OpenAI API key, Codex login, hosted Kea account, or other credentials. Judges can clone the repository, run npm run demo, and inspect the generated report locally.
What we learned
The biggest lesson was that creating a plausible summary is much easier than creating one that can be trusted.
A model can return perfectly valid structured data and still make claims that its citations do not prove. Structured output and evidence validation are separate problems, and Kea needs both.
I also learned how easy it is to confuse activity with success. A session can contain many commands, edits, and confident explanations without showing that the intended feature actually worked.
The project also helped clarify how I work with different AI tools.
GPT-5.6 was most useful for product reasoning, planning, defining milestones, and reviewing decisions. Codex was most useful for repository exploration, implementation, testing, and debugging. My role was to decide what to build, review the work, and step in when the implementation moved outside the intended scope.
Recording the session activity was only the first step. The harder part was deciding what I could honestly say had happened.
What's next for Kea
The current hackathon MVP is focused on Codex and runs from the cloned Kea repository.
The first practical next step is making Kea installable inside an existing project through a setup flow such as kea init or npx kea init.
Other possible next steps include:
- support for additional AI coding agents.
- a more polished installation and onboarding flow.
- make it a public npm package
- optional sharing of validated, sanitized reports with a team.
- organization-managed installation and hook policies.
- stronger secret detection and privacy controls.
- better handling of large sessions.
- and more validation rules for different types of engineering claims.
A future team version could optionally synchronize validated reports to a hosted project dashboard while keeping raw session recordings local.
Kea's purpose would stay the same... show what the coding agent did, what was verified, and what is still uncertain.
Log in or sign up for Devpost to join the conversation.