Inspiration
Every other engineering discipline preserves its reasoning alongside its output. A bridge's load calculations and failure assumptions are kept for the life of the structure — an inspector decades later can still ask "what was this built to withstand?" Software is the only engineering discipline that discards its reasoning the moment it compiles. The assumptions that made a function correct — this list is never empty, this pointer is never null, this only runs on one thread — live only in the original author's head, for as long as they remember, and then nowhere.
We realized this has been misdiagnosed for decades as a documentation problem, when it's actually a verification problem. Documentation describes code; it doesn't check whether the code still honors what it always silently depended on. That gap — not syntax errors, which compilers already catch — is the real source of most of the confusing, hard-to-trace bugs and outages that show up in production.
What it does
Axiom Drift analyzes a public GitHub repository and:
- Extracts candidate functions from the source.
- Uses an LLM to identify each function's implicit assumptions — things it silently depends on being true but never checks — as structured, falsifiable claims (not prose).
- Traces real call sites of that function elsewhere in the same codebase.
- Uses an LLM to judge whether each call site actually satisfies, violates, or leaves unclear each assumption.
- Renders the result as an interactive schematic graph — function nodes connected to the real call sites checked against them, color-coded by verdict — so you can click straight into the exact place a hidden assumption quietly breaks.
The output isn't documentation. It's a live map of where a codebase's stated and actual behavior have drifted apart.
How we built it
The pipeline runs entirely client-side: it fetches a repository's file tree and contents through the GitHub API, extracts functions with a lightweight heuristic parser, and sends each function to Claude with a strict JSON schema prompt to extract implicit assumptions as structured claims (type, target, condition, confidence) rather than free text. We then search the fetched codebase for real call sites of each function and send each one back to Claude alongside its assumptions, asking for a verdict — satisfies, violates, or unclear — grounded in the actual visible code. Everything renders live into an SVG schematic graph built from deterministic radial geometry, so the diagram updates node-by-node as the analysis streams in.
Challenges we ran into
- Keeping the LLM's output strictly parseable JSON rather than prose, since a single malformed response could break the pipeline mid-run.
- Designing a call-site detection approach that's honest about its limits — we use text-based tracing rather than a full call graph, which is fast and dependency-free but can miss indirect or dynamic calls.
- Deciding how much rigor to claim. It was tempting to oversell this as "verification," but we were careful in both the build and the paper to be precise: this produces strong evidence of assumption drift, not a formal proof — and we think stating that limitation honestly is more credible than overclaiming.
- Keeping the demo fast and legible by deliberately capping the number of files and functions analyzed per run, rather than trying to brute-force full repository coverage in a week.
What we learned
That the real unlock here isn't "AI can read code" — everyone already knows that. It's that an LLM's ability to describe a function in prose can be forced into a structured, checkable claim, and once an assumption is structured, it stops being descriptive and becomes falsifiable. That's a small reframing with a large consequence: it turns "explain this code to me" into "does this code still believe what it originally assumed" — which is a fundamentally different, and much more useful, question.
What's next for Axiom Drift
As AI systems generate a growing share of the world's code, the human safety net — someone who understands why the code was written a certain way — is eroding. We see this project as a first step toward software that carries a living, falsifiable model of its own reasoning: replacing full-codebase AST-based call graph tracing (instead of text search), extending beyond Python, adding CI integration so drift is caught automatically on every pull request, and moving from LLM-judgment verdicts toward hybrid static/LLM verification with higher confidence guarantees.
Log in or sign up for Devpost to join the conversation.