DriftGuard

AI-built apps are everywhere. DriftGuard generates invariants from your code, catches silent corruption, reports it in plain English your family will trust - and heals it.

Comment

Inspiration

Most people seem to be building more personal apps with AI tools. My pace has gone up - menu planners, trackers, family utilities. But the truth is that about half died when I returned to them months later, and roughly 60% of the ones my family could have used never got shared - mostly because I couldn't promise they'd keep working.

I don't worry too much about the app crashing. But the silent ones worry me - like the sync bug that quietly corrupts data for weeks, nobody notices, but then someone who didn't build the app — a spouse, a kid — hits a blank screen. They don't file a bug report. They just stop using it for good.

What it does

DriftGuard is an immune system for AI-built apps:

  • driftguard scan — GPT-5.6 reads your app's schema and source code and generates the data invariants that were not written down: referential integrity, freshness, volume bounds, and - the interesting part - domain rules inferred from the code's semantics (e.g. "a dinner slot's recipe must belong to the same household").
  • driftguard check — runs the invariants against your database and renders a plain-English weather report: sunny (healthy), cloudy (something's off, your data is safe), stormy (needs the builder). No stack traces. It's written for the family member who uses the app, not the person who built it.
  • driftguard heal — on a breach, GPT-5.6 gets a bounded repair task: propose a single SQL transaction, apply it to a sandbox copy, re-run every invariant, and only promote the fix if everything passes. Break it, watch it notice, watch it fix itself.

The core insight: the AI that built your app is the only thing that can credibly write the assertions about it. DriftGuard makes it do exactly that.

Try it on your own app: driftguard scan <your-app-dir> -o mine.yaml && driftguard check -c mine.yaml - any directory with a SQLite database and its source code. Watch it infer the rules that weren't written down. (A second bundled example, examples/chore-chart, is included exactly for this - its invariants were never hand-checked before shipping.)

How we built it

Entirely with Codex and GPT-5.6, in a single Codex project thread, from an AGENTS.md spec written before the first prompt - scope, stack, build order, out-of-scope, and working agreements. Then I ran it as goals with completion proofs: each milestone had to prove itself ("scan produces all five invariant classes and the YAML round-trips through the checker's parser") before we moved on. Codex tested, fixed, and continued until each proof passed - the same review-repair-validate loop DriftGuard now runs against your data.

The pace surprised me: scaffold + demo app + scan in 21 minutes, check + weather report + demo-break in 26, the full heal engine in 14. Codex delegated the demo app and test authoring to subagent threads while keeping the core in the main thread, committed in small evidence-point commits, and worked through real obstacles on its own - a GitHub SSH blocker (solved via device auth and a dedicated deploy key), a Node 17 machine below the required Node 20 (switched to its bundled runtime), and instead of guessing the current Responses API structured-outputs contract, it went and read the official docs.

My job was the product decisions - and the distrust. Which turned out to matter (see challenges).

Challenges we ran into

The tests passed, but the product was unproven. Codex's tests all ran against a mocked model because no API key was configured, which meant GPT-5.6 had never actually generated an invariant. Since invariant quality was the whole bet, I stopped building and ran the first live scan before writing another line of code. It caught the cross-household recipe rule on the first try.

It worked on my machine. To check the repo would hold up for anyone reviewing it, I cloned it to /tmp and followed my own README like a first-time user. That surfaced two things the 19 passing tests never could: the CLI didn't load .env (fine in tests, broken for anyone else), and a port clash dumped a raw Node stack trace in a product whose whole pitch is "no stack traces."

The model isn't deterministic. My scan generated 9 invariants; the fresh clone's generated 7. Both caught all three demo corruptions, but an unlucky scan during judging might not have. The fix Codex built was better than what I asked for: instead of checking that specific invariants exist by name, it verifies behavior, running each fresh scan's set in a sandbox to prove it actually detects all three corruptions, with one retry and bundled fallbacks merged only if something's truly missing. Generation stays live; the demo stays reliable.

Making self-heal something you can trust. Letting a language model repair your data is exactly the kind of feature that demos beautifully and wrecks things in production. So every guardrail is there on purpose: repairs run in a sandbox first, nothing touches real data until every check passes, two attempts max, a backup before any write, and a flat refusal to run outside the demo app. The alternative is magic you can't trust.

Accomplishments that we're proud of

  • Heal that refuses to overreach: sandbox-first, full invariant re-check, max two attempts, and it only operates where it's been proven safe. Honesty over magic.
  • A failure report a non-technical family member can actually read.
  • The same unwritten rule, re-inferred with identical SQL in every independent scan - the capability is repeatable, not a demo fluke.

What we learned

The headline: there's a real difference between a model that writes code and a model that reasons about someone else's code's intent. The cross-household recipe rule isn't in any schema - GPT-5.6 read the app and understood what it meant. And it wasn't luck: across independent scans it re-derived the same rule every time, with identical SQL.

Second: model variance is a product-design problem, not a bug to file. You don't pin the output - you verify the behavior.

Third, my favorite: when I told Codex to fact-check the README against the codebase before shipping, it dialed the claims down - corrected an overstated heal description, reframed a lucky demo moment as an observed result rather than guaranteed behavior.

What's next for DriftGuard

Postgres support, watch mode on a schedule, and heal for arbitrary apps once the guardrails deserve it. The dream: every AI app-builder ships a driftguard.yaml next to its README.

Built With

Share this project:

Updates