Vaulter

Inspiration

I've worked with senior software architects and it taught me one thing: good engineers ask different questions. Every time I brought them an idea, they killed it in a few minutes, and for the better. It rubbed off on me.

AI has made building cheap but not architectural judgment. Non-technical builders prompt their way to a working demo in an afternoon, and it holds up right until it meets a real user base. The reason is simple, they start wrong. They open with the feature and UI/UX design list. A senior engineer opens with a different question: what must always be true here, and what must never happen? Vaulter asks that question for them.

What it does

Vaulter asks the user 5 questions in plain english before starting a new project:

1. Does money move?
2. Can users see each other's data?
3. What can a stranger with no account do?
4. Does it call an AI model, and who pays?
5. What would you hate to see leaked?

Answers run against a rubric built from canonical engineering practice and compile into invariants, hard rules like a payment webhook delivered twice must never grant two subscription periods.

Then Vaulter attacks its own output. A second adversarial pass reads the finished spec as a hostile reviewer. On the test build it found a race condition where concurrent requests each pass the same spend check and blow through the budget cap, a failure invisible to single-request testing, and it wrote the test that catches it.

You get three files: a Build Contract, an AGENTS.md your agent keeps in context, and acceptance tests to verify it obeyed.

How I built it

I researched books and articles using ChatGPT from reputed sources (Meyer's design by contract, Lamport's safety properties, OWASP's API and LLM Top 10, Nygard on architecture decision records, etc..) for universally accepted engineering principles to prepare a questionnaire that is presented in front of a user. Each question maps to invariants, irreversible decisions, and tests. The rubric decides what applies. The model never improvises that. Then there are two passes: Pass one generates under a strict JSON schema. Invariants with rationale, expensive-to-reverse decisions, omissions with an owner named, given/when/then tests. Validated structure, not prose I hope parses. Pass two is adversarial. A separate call gets the finished spec and a hostile-reviewer role: assume the author was rushed, find what's missing, focus on concurrency, authorization, and anything that costs money. Every invariant it adds ships with a test. Two calls, opposed objectives, the second grading the first. Front end is a studio in Next.js. Questions on the left, live repo structure and contract stream on the right, driven by the model's actual structured output.

Challenges I ran into

I was very happy when the pilot version produced a file with a decent master prompt, but then I asked myself, how would a non-technical founder verify that this file is actually technically sound? If he does not know what to ask, he certainly does not know what is right or wrong. Built the second pass for that. I initially had a live typing effect that became a buffered replay of a real result, it traded it for validated structure. Tuning the adversary was also tricky, if it is too soft it wont return anything and if too hard, it invents faulure modes. Limiting it to concurrency, authorisation and cost made it find real bugs. Still need improvement though.

Accomplishments that I am proud of

I applied the Vaulter's own rubric to build Vaulter, and it showed me some serious loopholes, philosophical and technical, which could have seriously damaged the tool's validity. Hence the product ate its own dogfood to improve and produce defensible results.

What I learned

Tools can become fast, but there is no replacement for human judgment, and that is the zone where the skill needs to be polished for the coming developers. An AI will always pass the tests it wrote itself, and that is why it needs a human that questions and verifies the test itself.

What's next for Vaulter

Two things:

1. The blueprint layer. Right now it tells you what must not break. Next it will tell you what you are building, so that the invariants have something concrete to attach to.
2. The runner. If one points vaulter at a deployed URL, it will attempt the impossible things, read across accounts, replay a webhook 20 times, fires concurrent requests at a spend cap. This way it turns a document to a verdict that you can verify.

Built With

• codex
• google
• gpt
• node.js
• vs