ChangeGuard

• 

  home page

• 

  home page

• 

  export page

• 

  diff page

Inspiration

I use AI coding agents regularly, and I also spend time writing rules and skills to guide how they work. While using them, I noticed that some of the worst mistakes are easy to miss. An agent might remove a test assertion, skip a test, or add an empty catch block that hides an error. The code can still look perfectly fine, and the tests might even pass.

Reviewing large diffs generated by AI agents takes time, especially when the risky changes are buried among dozens of normal changes. I also found it difficult to trust AI code review tools that explain potential problems without making it clear which findings are backed by actual evidence.

That’s why I built ChangeGuard: a tool that helps developers review AI-generated code changes and understand what was detected by a rule, what is only a possibility, and what still needs to be checked.

What it does

ChangeGuard lets you paste a Git diff containing JavaScript or TypeScript changes and reviews it in two ways.

The first is rule-based analysis, which runs directly in the browser without using an LLM. It currently checks for removed test assertions, empty error handlers, and skipped or focused tests. These findings have solid blue borders because they are based on patterns found directly in the diff.

The second group contains hypotheses generated by an LLM. These appear with dashed amber borders and are always labeled as uncertain. The model returns line IDs rather than code quotes, and ChangeGuard retrieves the corresponding lines from the actual diff. If a finding references a line that doesn't exist, it is moved to a separate section for claims without valid citations.

A finding is never automatically treated as a confirmed bug, and an empty result doesn't mean the code is safe.

You can click a finding to jump to the relevant line in the diff, then export a pull request comment or an agent prompt that asks the coding agent to verify the suspected issue with tests rather than blindly changing the code.

How we built it

The application is designed to run 100% locally in the browser using a Web Worker and pre-cached evaluation fixtures, ensuring tests and verification run instantly with zero network dependencies or API key exposure. The application performs its rule-based checks client-side, with support for live LLM proxy integration designed to fail gracefully to rules-only mode.

Challenges we ran into

One of the biggest challenges was deciding how to describe findings accurately. Initially, it was tempting to call a finding verified when its citation had been checked. But verifying that a line exists doesn't prove that the model's explanation is correct. I changed the labels and explanations to make that distinction clear.

Writing the detection rules also required more care than expected. For example, a simple pattern for skipped tests such as xit( can accidentally match process.exit(. Similarly, matching expect( can produce false positives for calls such as .expect(200) in Supertest. I had to refine the patterns and add tests for cases that should not trigger a finding.

Another challenge was making sure the two analysis paths remained independent. The deterministic rules should continue working even if the LLM is unavailable, and model-generated findings should never be allowed to create their own evidence or change the labels assigned by the application.

Accomplishments that we're proud of

I'm proud that the separation between rule-based findings and LLM hypotheses is enforced in the implementation, not just represented through different colors in the UI.

The rule-based checks work without an LLM, and the citation checker ensures that displayed code references come from the actual diff. Invalid references are separated for review instead of being presented as trustworthy evidence.

I also tested ChangeGuard against real diffs from my own AI coding sessions and committed the evaluation results to the repository, including findings that were noisy and cases where the tool missed issues. This was important because I wanted to understand the limitations of the tool rather than only showcase successful examples.

The adversarial fixture also gave me a way to test how the system handles a diff containing an instruction intended to influence the model's review. In my recorded test, the model still identified the regex-based XSS risk, while the deterministic rules remained independent of the model. The fixture also demonstrates how a claim with a nonexistent line reference is handled.

What we learned

The biggest lesson was that reviewing a diff has limits. Some problems depend on code outside the changes being reviewed, so a tool that only sees the diff cannot always know whether a change is safe.

I also learned how much difference a clear plan makes when working with AI coding agents. Defining the scope and requirements before implementation helped me guide the agent and catch misunderstandings earlier.

Finally, communicating uncertainty turned out to be a design challenge of its own. It's easy to build a warning, but much harder to explain exactly what that warning proves and what it doesn't.

What's next for ChangeGuard

There are several areas I'd like to improve next:

• Detect weakened test assertions, such as changing toBe(5) to toBeTruthy().
• Detect entire deleted test blocks, rather than only removed assertion lines.
• Add support for Python, Go, and Rust.
• Build a GitHub Action so ChangeGuard can review changes automatically on pull requests.
• Add repository context so the tool can inspect related code and callers instead of relying entirely on the diff.

Built With

• antigravity
• css3
• git
• html5
• javascript
• preact
• typescript
• vite
• vitest
• web-workers