DeceptiForge

DeceptiForge creates context-aware decoys across secrets, data, documents, and AI workflows—turning interactions into sensors that detect attacks, insider risk, and accidental AI leakage.

Comment

DeceptiForge

Context-aware deception for the AI era

What inspired me

Most deception tooling still ships the same primitive: a lone fake credential dropped into a config file, waiting for someone to grab it. That works against a script grepping for AWS_SECRET. It fails against anything that reads context first, and increasingly the thing reading your repository is an AI agent that notices a stripe_key sitting in a folder with no billing code, no imports, and no history. An isolated honeytoken is a decoy that does not fit its own scene.

I wanted the decoy to belong. If a payments repository gets a synthetic asset, it should carry that repository's vocabulary, live where a real engineer would have put it, and read as plausible enough that an agent exploring the codebase treats it as signal rather than noise.

That reframing, from fake secret to believable synthetic business asset, is the whole idea. Everything else follows: infer the context, generate to fit, place where it is plausible, and detect the touch through a real monitoring pipeline instead of a mocked one.

What I learned

A security header can break your application in complete silence. I added a nonce-based Content-Security-Policy and the entire app stopped hydrating. No console error. No failing test. A statically prerendered page is built once with no nonce, while middleware issues a fresh one per request, and under strict-dynamic a nonce is the only thing that authorises a script, so 'self' stops applying. Every script was refused and React never took over. My first verification was wrong too: I read server-rendered form inputs as proof the client was alive.

The only trustworthy signal turned out to be comparing the nonce in the served HTML against the one in the response header. That comparison is now a CI check.

Expiry checks are security checks, and they fail like security checks. Writing sandbox tests surfaced a latent bug: authenticating an expired API key compared a naive stored timestamp against an aware one and raised TypeError. An expired key returned 500 instead of 401. Harmless until judge credentials, which always carry an expiry, made it reachable.

Documentation that contradicts enforced behaviour is an operational defect, not stale prose. Several documents told operators the opposite of what startup validation enforced: REDIS_FAIL_MODE=open described as permitted, disabled auth described as merely rejected per request when it now refuses to boot. An operator following them would produce a deployment that does not start. I reversed an earlier architecture decision record rather than rewriting it, because an ADR records what was decided and why it changed.

Configuration leaks through .env. Enabling one feature flag locally broke six unrelated production tests, because pydantic-settings reads that file and the value reached assertions that construct settings directly. Deriving the hardened-mode set from a single exported constant, instead of a copy-pasted literal, removed a whole class of drift.

The through-line: in a security tool the dangerous failures are the quiet ones. The work was not making features exist. It was making the guarantees provable, on real HTTP responses, against a running system, in CI.

How I built it

A monorepo: FastAPI backend, Next.js dashboard, Plasmo browser extension, shared contracts package. The pipeline is a straight line you can watch end to end:

scan → context → placement → generation → validation → monitoring → alert → incident

Analysis is deterministic. It runs over bounded structured signals — languages, frameworks, services, naming patterns, sensitive-zone categories, AI surfaces — never raw source. Path-like values are descriptive metadata; the backend never opens them. Every inference reports its supporting evidence and a confidence value, so a reviewer sees why, not only what.

Detection is signed and fails closed. Monitoring ingestion uses an HMAC scheme with a replay-nonce store and organization-scoped rate limiting. A touched decoy emits a signed event; the event becomes an alert only after verification; the incident is reconstructed from stored evidence rather than asserted. Severity and timeline come from application logic.

GPT does two bounded jobs. It drafts synthetic content matching the inferred context, and turns a verified timeline into analyst-readable prose. It decides nothing that matters: not severity, not deployment, not whether an incident occurred. When the model is unavailable a deterministic fallback runs, and the interface labels it as such.

Calibration uses Wilson score intervals rather than raw acceptance fractions, so a decoy accepted 3 of 3 times does not outrank one accepted 40 of 45. For observed acceptance $\hat{p} = x/n$ at confidence $z$, ranking trusts the lower bound

$$ w^{-} = \frac{\hat{p} + \dfrac{z^{2}}{2n} - z\sqrt{\dfrac{\hat{p}(1-\hat{p})}{n} + \dfrac{z^{2}}{4n^{2}}}}{1 + \dfrac{z^{2}}{n}} $$

so small samples are penalised honestly and a cohort needs real evidence before it moves a weight. Anti-poisoning thresholds cap any single actor's contribution, and no learned weight reaches production without human approval.

Deployment modes let one codebase serve as local development, a hosted evaluation sandbox, or a locked-down production tenant. APP_ENV is a closed set, so a typo fails startup instead of silently selecting middle behaviour. The hosted judge mode is production-hardened and differs only in which demonstration surfaces mount. Each evaluator receives a TTL-bound sandbox with its own generated organization, so no one can see another's work.

Codex worked alongside me across implementation, migrations, tests, tenant-isolation and signed-ingestion review, CI debugging, and hardening. It was genuinely fast at mechanical breadth: wiring a router, drafting a migration, generating negative-path tests. It decided nothing load-bearing. Product direction, security boundaries, human-approval requirements, and final acceptance stayed mine. The pattern that earned its keep was adversarial: Codex proposes, I verify against the running system before believing it.

Challenges I ran into

One symptom, three unrelated causes. The judge workspace froze on its loading state. First cause: the nonce CSP blocking prerendered scripts. Second: next dev runs hot reload through eval and emits inline scripts with an empty nonce, and because a CSP containing a nonce makes browsers ignore 'unsafe-inline', my first fix did nothing at all. Third: two development servers sharing one build directory clobbered each other's chunks, so the page rendered correctly on the server and never hydrated. Identical symptom, no console error in any of the three. I found the last one only by counting attached React fibers between two servers running the same code: 21 on one, 0 on the other.

Making the demo safe to host. The demo routes had no authorization at all. Fine while they were development-only; the moment a hosted evaluation mode could mount them, five mutating endpoints became unauthenticated writes on an internet-reachable deployment. Splitting reads open and writes credentialed took more thought than expected: requiring a credential to look at a fixed fictional story buys nothing, but letting anyone reshape what every other viewer sees is a real problem.

Proving isolation instead of asserting it. It is easy to claim a sandbox cannot read files. Demonstrating it took a test that writes a marker into a real file, passes its path as a descriptive signal, and asserts the marker never appears in the response. If the backend had opened it, the analysis could leak the contents. Most of the isolation suite is shaped that way, and nearly all of its assertions are negative.

Knowing when my own verification was insufficient. More than once I declared something working on evidence that did not support the claim. Server-rendered HTML mistaken for hydration. A contract test that silently went hollow after a refactor moved the string it matched. Catching those mattered more than any single feature, because a security tool whose guarantees are unverified is just a tool with confident documentation.

What is next

Repository and context metadata is still plaintext at rest while alerts and incidents are encrypted. Closing that gap is the next real change. Beyond it: a hosted deployment with restore-drill evidence, and widening the calibration engine from bounded demonstration toward measured, human-approved production use.

Built With

Share this project:

Updates