DecisionTrace

Ask why a decision was made in your repo. Tell it the reasoning changed. It saves that correction, survives a restart, and uses it to answer every question after.

Comment

Inspiration

Most of us have hit this before. You open some code, see a decision that looks wrong or overcomplicated, "fix" it, and later find out it was reverted from your exact version for a reason someone already wrote down and nobody read again. The reasoning isn't gone, it just isn't searchable. It's sitting in a PR description or a KEP section that nobody reopens before making the same call twice.

We also asked ourselves the annoying question early on: why not just paste the PR into an LLM and ask? For one question, that works fine. It falls apart the second time someone asks the same thing and gets a slightly different answer, or when someone tries to relitigate a decision and there's no record that this exact argument already came up and got rejected. A regular chat has no memory of who asked before, what they were told, or what's already been tried. We wanted something closer to a coworker who's been around for years and remembers being argued with, and didn't cave.

What it does

DecisionTrace answers "why was this made" and "what actually governs now" against a real GitHub repo, and every claim links to a real PR you can open yourself. Ask it something like "why was delayed preemption reverted in kubernetes" and it comes back with two things: a verified historical fact tied to the PR that first implemented the change, and a current active decision tied to the PR that reverted it.

Open the agent trace and you get an actual record, not a disclaimer. It shows which agent ran, what it found, and what it decided to keep. Scroll down and you see the full lifecycle between the two PRs, plus a list of every past attempt to reconsider this exact decision. In our demo that list already had six entries on it before we touched anything, each one labeled proposed, not accepted, still sitting there.

You can also tell it an assumption changed and ask it to record a reconsideration. It writes that down right away, gives it its own id, and shows you a confirmation. What it does not do is quietly change the answer. Ask again later, from a different session, and it still remembers every one of those attempts, yours included, and still tells you honestly whether anything actually changed.

How we built it

The ingest side pulls real data, not fixtures. It goes into a GitHub repo's history, grabs merged revert PRs and KEP style "alternatives considered" sections, and uses Gemini to pull out the rationale, the evidence, and how decisions relate to each other.

Answering a question runs through four agents instead of one prompt. An evidence scout finds decisions that might be relevant. A lifecycle resolver replays them against typed edges like reverts and implements to work out what's actually governing right now, and this part is deterministic, same input always gives the same answer. A provenance challenger throws out anything that doesn't have real evidence behind it. Then a reconciler writes the actual answer, but only using what already survived the challenge step, and it checks its own claim ids against what got approved.

The part we spent the most time on is how a reconsideration is stored. Recording one creates a new decision with a reconsiders edge pointing at the thing it's challenging. The key part is that reconsiders is deliberately left out of the edge types that can flip which decision is active. So no matter how many reconsiderations pile up, or how convincing they sound, the resolver physically cannot let one silently become the new answer. They stay visible, marked proposed and not accepted, until someone adds a real acceptance step, which we left out on purpose so a proposal can never be confused with an approval.

It's live on Cloud Run with Firestore behind it, so "remembers across sessions" isn't something we're claiming, it's just what happens when you open it twice.

Challenges we ran into

The hard part wasn't retrieval, it was making sure a reconsideration could never quietly become the accepted answer. The easy way out would've been letting the model decide if an argument was good enough to accept, but that just brings back the exact problem we were trying to avoid, a good sounding argument overwriting the actual history.

We also didn't want to just claim this beats a plain LLM conversation without checking. So we went and grepped the whole codebase for the specific candidate ids showing up in the live app, since those ids are just a hash of the target and the exact text someone submitted. Nothing matched anywhere, not in the source, not in tests, not in any fixture. That's how we know those six reconsiderations in the demo are real ones from actual use, not something we seeded to look good.

Accomplishments that we're proud of

The decision in our demo really has survived six separate attempts to overturn it, and it's still standing, which we only found out was true by going and checking, not by assuming. Every claim it makes points to a real PR you can click into and read yourself.

What we learned

The thing that actually matters here isn't a bigger model, it's memory that persists and a rule that can't be talked around. A one-off chat has no idea who asked this before or what they were told, and nothing stops a good argument from changing the answer next time. Putting that rule into the data itself, not the prompt, not a UI check, is what makes it hold up even in cases we didn't think of ahead of time.

What's next for DecisionTrace

Add the actual acceptance step we left out, so a proposed reconsideration can become the real governing decision with its own record of who approved it and why, without losing the guarantee that a proposal alone can never do that on its own. After that, pull in more sources than just revert PRs and KEPs, and let this work across more than one repo at a time.

Built With

Share this project:

Updates

Submission history