FORGE

AI changed who can build software. FORGE is my attempt to change how we learn to trust it. | A forensic governance runtime for software audits you can reproduce, inspect, and challenge.

Comment

FORGE

AI changed who can build software. FORGE is my attempt to change how we learn to trust it.

The project

Hi, I’m Anna.

AI removed the syntax barrier that had kept me from building software. Since then, I've built open-source systems for legal, cryptographic, forensic, and semiological applications.

My focus is architecture, invariants, failure modes, and traceability. I care about building systems that remain trustworthy over time.

Before software, I was a professional cook, and I am transferring that gastronomic knowledge and process discipline into software. It may sound unrelated, but the methodology is remarkably close. In a professional kitchen, washing your hands is not optional, and neither should basic IT security be. Rotten ingredients are discarded instead of being hidden under expensive spices. Mise en place exists because people under pressure make mistakes. Traceability exists because when something goes wrong, you need to know exactly what happened.

Software deserves the same discipline.

Why FORGE exists

Over the past few years, I've built more than thirty systems with the same commitments: auditability, bit-for-bit determinism, exact arithmetic with Fraction instead of floats in decision paths, explicit invariants, and security-first architecture. I believe software deserves long-term stewardship, not abandonment.

FORGE was not designed as a hackathon idea. It emerged because maintaining more than thirty deterministic systems exposed the same engineering problem over and over again: building software is becoming easier every month; maintaining justified confidence in it is not.

I've always gravitated toward Purple Team work. I can't simply watch attack surfaces accumulate. Instead, I deliberately red-team my own systems. That is necessary, but it takes a great deal of time — especially when the problems are not obvious bugs that an agent can solve with “find the vulnerabilities.”

FORGE focuses on architectural failures, broken invariants, and epistemic flaws — the kinds of problems a generic “find vulnerabilities” prompt rarely understands.

FORGE is not a disposable hackathon project. It is a system I already use and will continue improving. It applies a scientific workflow to software audits:

Observe → Hypothesize → Falsify → Verify → Seal

FORGE does not treat every suspicious pattern as a bug. It builds hypotheses, tries to prove them wrong, preserves discarded ideas and scope limits, and produces evidence another investigator can inspect.

Evidence at hackathon scale

FORGE is not backed by one polished demo report.

This repository includes 46 MB of sealed HTML and JSON audit artifacts in results/. The companion public forge-results archive contains another 764 MB.

Together, that is over 800 MB of reproducible audit evidence — approaching a gigabyte: self-audits, real-repository case studies, benchmark runs, sealed manifests, traces, discarded hypotheses, confirmed findings, false positives, unresolved leads, and regression-backed fixes.

The archive exists so judges can inspect FORGE as an accumulated forensic engineering record, rather than taking one or two showcase outputs at face value.

Four execution modes

The same audit runtime can be used through four interfaces:

  • CLI for reproducible local audits and sealed reports.
  • Python API for embedding the audit runtime into engineering workflows.
  • MCP for connecting FORGE to an agent through a governed tool boundary.
  • Multi-agent orchestration for comparing external work products against FORGE’s own evidence and contracts.

CLI and Python run locally with zero tokens and zero internet dependency. MCP and multi-agent modes add bounded external validation with model routing that is explicit and recorded honestly. The runtime never claims a model participated when it did not.

When invoked, GPT-5.6 complements FORGE’s investigation. It does not replace it or become its authority.

It can inspect what FORGE did, which hypotheses it tested, which it discarded, whether the evidence agrees with the code, and what a human should verify or fix next. From that evidence, it can help produce a reviewable remediation plan; it cannot silently turn a hypothesis into a bug or patch the repository by authority. The deterministic runtime remains the source of the audit result.

Twenty reusable engineering skills

FORGE does not rely on clever prompts. It includes twenty governance skills that make engineering practice reusable for both experienced developers and people learning through AI. Those skills are reusable engineering methods, not prompt templates. They can be applied by humans, AI agents, or both:

  • Core reasoning: abductive engineering, red-team auditing, secure by construction, software archaeology, diagnosing bugs, codebase health, reverse engineering, and Daubert-defensible writing.
  • Determinism and integrity: deterministic core, LLM out of the loop, tamper-evident audit chain, atomic state mutation, and versioned schema evolution.
  • Patching and editing: surgical patcher and audit before patch.
  • Input and data: validate at the boundary, honest degradation, and SQL aggregation rather than materialization.
  • Process and evidence: git discipline and claim provenance discipline.

Prompts do not compound. Engineering practice does. These skills do not merely teach an agent what answer to produce; they teach it how to reason, preserve evidence, patch safely, and admit uncertainty.

Full catalog: https://github.com/annatchijova/forge/tree/main/skills-gpt

False positives are architectural signals, not noise

FORGE produces false positives. That is not a defect; it is a deliberate epistemic boundary.

A generic bug finder discards anything that does not match a known pattern. FORGE does the opposite: it preserves the signal, documents the surrounding mechanism, and leaves the exact scope and limits so a reviewer can investigate. In a read-only audit of VIGÍA, a contextual false positive was not erased. Reviewing the mechanism around that signal led to a separate, complex, reachable defect that no detector had flagged directly. That defect was independently verified, fixed, committed, and pushed.

FORGE’s original signal remains exactly what it was: a lead with documented limits. It was not retroactively rewritten as a “true positive” to make the report look cleaner. The report preserves what was actually proven, what was not, and what remains unresolved.

That is the behavior an audit system should have: a signal you cannot fully verify is still valuable if it forces the right architectural question, but the report must never pretend certainty it did not earn.

CRONOS and the cost of losing provenance

FORGE also includes CRONOS, a tamper-evident audit and reasoning trail, exposed through the FORGE MCP tooling alongside the FORGE-loop MCP.

During development, an AI agent nearly destroyed months of my mother’s machine-learning work. The agent had been instructed to remove every float, but this system depended on ML behavior that had to be understood before changing it. CRONOS made the agent’s path and reasoning visible in the audit trail, so the destructive action could be understood and stopped instead of becoming an unexplained loss. I also used the same MCP infrastructure in another project with excellent results.

The goal is not to eliminate mistakes. It is to make mistakes observable before they become irreversible.

Proof through use

I have exercised FORGE across dozens of repositories and controlled experiments. The preserved forge-results material contains audit artifacts, reports, false positives, confirmed findings, and known limits. The repository also includes a results collection for judges to inspect:

https://github.com/annatchijova/forge-results

https://github.com/annatchijova/forge/tree/main/results

FORGE found a real infinite loop in its own Integrity Inspector. The problem was triggered by an ordinary branch pattern with different float() reassignments. It was fixed with a regression test; the same audit now finishes in roughly 2.5 seconds instead of hanging beyond a 280-second timeout.

How Codex and GPT-5.6 helped

Codex and GPT-5.6 helped me turn a working methodology into a usable product:

  • repository archaeology across a growing Python codebase;
  • test-driven hardening and regression design;
  • report and evidence-package UX;
  • MCP and multi-agent boundaries;
  • documentation, demo preparation, and judge-facing artifacts;
  • adversarial review of the runtime’s own assumptions.

The collaboration loop was deliberately evidence-first and token-efficient: local deterministic audits, tests, static checks, and repository inspection performed the first pass; Codex then used ChatGPT with GPT-5.6 to review the failure and the relevant diff, validate whether a suspected bug was real, and start a bounded correction only when the evidence supported it. This kept model context focused on difficult reasoning and repair instead of spending tokens rediscovering issues that local tooling could already narrow down. Every correction was then checked against the live code and regression suite.

The model did not become the authority over findings. FORGE’s local runtime still controls discovery, scope, detector execution, hypothesis handling, canonicalization, sealing, and verification. ChatGPT can review that work and help produce a remediation plan, but the evidence and the human investigator remain responsible for the final claim.

Who FORGE is for

If you work on a complex system — legal, cryptographic, forensic, ML, infrastructure, or security-sensitive — FORGE gives you a way to audit the architecture instead of merely searching for obvious patterns.

If you are a senior developer, it can make deep review reproducible and traceable. If you are learning through vibe coding, the MCP runtime may be more than you need at first; the reusable skills, the engineering methodology, and the governed loop can teach you practices that attractive prompts cannot.

Open source

Closing

FORGE does not promise that every lead is true. It promises something more useful: that the search, the uncertainty, the discarded hypotheses, and the evidence remain visible.

AI changed who can build software.

FORGE is about making trust reproducible.

Skills:

FORGE — 20 Governance Skills

FORGE keeps 20 documented engineering and governance skills in skills-gpt/. Together they define how the runtime reasons, audits, patches, preserves evidence, and communicates uncertainty.

# Skill Category Activates when
1 abductive-engineering Core reasoning Debugging, root-cause analysis, incident response, or architectural decisions under uncertainty.
2 red-team-auditing Core reasoning Security audits, adversarial review, threat modeling, or invariant analysis.
3 secure-by-construction Core reasoning Writing, extending, refactoring, or reviewing code with security boundaries.
4 software-archaeology Core reasoning Modifying legacy, inherited, or unfamiliar code without breaking behavior.
5 diagnosing-bugs Core reasoning Investigating hard bugs and performance regressions through controlled probes and regression tests.
6 codebase-health-assessment Core reasoning Classifying dead, fossil, live, and out-of-scope modules before changing a codebase.
7 reverse-engineering Core reasoning Reconstructing undocumented systems, binaries, protocols, file formats, or opaque APIs without readable source.
8 daubert-defensible-writing Core reasoning Writing findings and reports that separate evidence, inference, uncertainty, and opinion.
9 deterministic-core Determinism & integrity Producing bit-for-bit reproducible and tamper-evident decisions with canonical serialization and SHA-256 sealing.
10 llm-out-of-the-loop Determinism & integrity Keeping consequential decisions outside the LLM path and sealing results before optional narration.
11 tamper-evident-audit-chain Determinism & integrity Building or verifying append-only logs that detect alteration, insertion, reordering, or deletion.
12 atomic-state-mutation Determinism & integrity Making multi-write persistent operations all-or-nothing and isolated from concurrent callers.
13 versioned-schema-evolution Determinism & integrity Evolving serialized artifacts with explicit schema versions without breaking existing data.
14 surgical-patcher Patching & editing Applying anchored, verified, reversible changes instead of rewriting entire source files.
15 audit-before-patch Patching & editing Validating an audit finding against the current file before changing any code.
16 validate-at-the-boundary Input & data Validating untrusted input at the system boundary with clear errors.
17 honest-degradation Input & data Handling degraded, legacy, reconstructed, or unverifiable input without returning plausible-looking wrong results.
18 sql-aggregation-not-materialization Input & data Pushing counts, sums, and grouping into the database instead of loading rows into memory.
19 git-discipline Process Keeping AI-assisted coding sessions recoverable, reviewable, and free from unsafe history rewriting.
20 claim-provenance-discipline Evidence governance Preserving each claim's origin, epistemic level, scope bound, and falsifier across summaries and handoffs.

Runtime status

The 20 skills are the documented governance catalog. The executable runtime currently has contracts for a smaller subset, including validate-at-the-boundary, honest-degradation, deterministic-core, atomic-state-mutation, sql-aggregation-not-materialization, and tamper-evident-audit-chain. The remaining skills are recorded as process-level obligations and methodology documents rather than being pretended to be per-module scanners.

Codebase Statistics

Forge

$ git ls-files '*.py' '*.md' '*.json' '*.sql' '*.html' | cloc --list-file=-

491 text files.
479 unique files.
12 files ignored.

github.com/AlDanial/cloc v1.98

-------------------------------------------------------------------------------
Language                     Files    Blank   Comment      Code
-------------------------------------------------------------------------------
JSON                           133       19         0    537,317
HTML                            42       82         0    185,577
Python                         233    2,138     2,066     11,603
Markdown                        71    2,471         1      9,149
-------------------------------------------------------------------------------
Total                          479    4,710     2,067    743,646
-------------------------------------------------------------------------------


**Forge-results**

$ git ls-files '*.py' '*.md' '*.json' '*.sql' '*.html' | cloc --list-file=-

616 text files.
563 unique files.
53 files ignored.

github.com/AlDanial/cloc v1.98

-------------------------------------------------------------------------------
Language                     Files    Blank   Comment        Code
-------------------------------------------------------------------------------
JSON                           401        0         0   5,097,559
HTML                           119      216         0   2,766,325
Markdown                        43      853         4       4,204
-------------------------------------------------------------------------------
Total                          563    1,069         4   7,868,088
-------------------------------------------------------------------------------

JUDGES: https://github.com/annatchijova/forge/blob/main/docs/hackathon.md

Built With

Share this project:

Updates