-
-
Same workload: stateless execution creates 36 skills; ForgeAgent creates 4 and reuses 32 verified capabilities.
-
Evaluation Arena: 50 deterministic cases passed and 10 unsafe capability proposals were rejected.
-
Runnable architecture: MCP requests flow through graph, Foundry, sandbox proof, governance, audit, and reuse.
-
Security regression proof: import-alias, dunder, dynamic-getattr escapes, and restricted imports are blocked.
-
Live trust-gate run: filesystem, network, eval, import-alias, and contract-violating tools are blocked.
-
A browser-first incident run: redact sensitive data, triage risk, extract terms, and record trusted lineage.
-
Offline Capability Foundry: invoice extraction is proposed, adversarially tested, sandboxed, scored, and promoted.
-
Repository intelligence maps code, docs, and tests so new capability proposals can avoid duplicate work.
-
Reproducible evidence: trust-gate attacks, 50-case evaluation, reuse comparison, and security regressions.
-
Offline multi-step autonomy: redact PII, triage the incident, then reuse a proven error extractor in order.
-
Privacy-to-risk runtime: ForgeAgent redacts PII, triages urgency, and extracts structured stack-trace evidence.
-
Forge once, verify once, reuse later: the second run uses trusted date normalization instead of regenerating code.
-
GPT-5.6 Terra's trusted v2 is reused on a later request—no rebuild and no new model generation.
-
GPT-5.6 Terra proposes an invoice extractor; adversarial proof rejects two candidates before v2 earns trust.
Inspiration
AI coding agents are increasingly trusted to write and run their own code. But generated code is often trusted before it has earned that trust: agents can rebuild the same unverified tool repeatedly, and a tool can look correct until someone deliberately tries to break it.
ForgeAgent asks a different question: what if an agent capability had to earn trust — through policy checks, isolated proof, and human approval where it matters — before it could be reused?
What it does
ForgeAgent gives coding agents a governed capability memory. When an agent needs a capability it does not already have, the Foundry Council plans it, builds a constrained proposal, checks it against the policy gate, proves it in an isolated sandbox, and routes sensitive work to named human approval.
After approval, a later coding agent can reuse the trusted capability from persistent project memory instead of rebuilding it. This is verified with a two-process regression test: one process requests and approves a capability; a completely separate OS process reuses it with memory_source: platform_store. python3 main.py --demo --reset # forge, prove, and trust a capability python3 main.py --demo # reuse it on the next run — no rebuild
How we built it
Capability memory + versions: SQLite-backed project memory records capabilities, proof evidence, governance decisions, receipts, and rollback history. Foundry Council: Planner, Builder, Security, Evaluator, and Governor roles produce a visible decision trail. Proof before promotion: static policy checks and sandboxed normal, edge, contract, and optional adversarial proof cases block unsafe or incorrect candidates. Repository intelligence: a repository graph finds related code before a new proposal is generated, helping avoid duplicate work. Developer integration: a local MCP server lets Codex, Cursor, and Claude Code inspect capabilities, request governed proposals, read receipts, and make approval decisions. Supply-chain controls: capability packages support Ed25519 signing, compatibility checks, revocation, and review-only import. Policy as code: forgeagent-policy.yml can only narrow the hardcoded safety baseline; it can never grant a broader permission.
Verified GPT-5.6 Terra run
A live OpenAI GPT-5.6 Terra run created a missing extract_invoice_ids capability from an explicit ASCII-only contract. The first two candidates were rejected with 5 and 3 proof failures; GPT-5.6 Terra generated adversarial cases across the repair loop. The repaired extract_invoice_ids@v2 passed normal, edge, three contract, and two adversarial cases at trust score 100. Repeating the exact task returned status: reused with correct new invoice IDs and no new model generation.
The live-provider path uses GPT-5.6 Terra for unfamiliar capabilities, semantic task matching, and adversarial proof when an OpenAI API key is configured. The hosted walkthrough, complete judge flow, benchmark, and evaluation remain deliberately key-free.
Challenges we ran into
The meaningful bugs were not the obvious ones. We found that generated code could alias import and evade a policy check that only matched direct calls. We closed it with two independent defenses: static AST detection of the name reference and a runtime-enforced import allowlist in the sandbox.
We also found a cross-process reuse defect: an MCP capability request could read the wrong memory store and rebuild instead of reusing. The regression now starts two real OS processes so the result reflects how a judge or team would actually use the system.
Accomplishments that we're proud of
Every important claim has a command behind it: python3 -m unittest discover -s tests -v # 87 passed; 6 optional policy tests skip without PyYAML python3 main.py --benchmark # 8/8 trust-gate cases pass python3 main.py --evaluate # 50/50 evaluation cases pass; 10 unsafe proposals rejected python3 main.py --compare # stateless: 36 new skills; ForgeAgent: 4 new skills, 32 reuses
The repository also includes a local live dashboard, an append-only JSONL audit log plus SQLite records, a judge-mode lifecycle that forges → approves → reuses → quarantines → repairs → reuses, and a GitHub Actions PR capability-review workflow that calls the same policy/proof code.
What we learned
The best proof is not a longer feature list. It is a system that survives an attempt to break it, preserves the evidence, and refuses to silently turn an unproven proposal into a reusable capability.
What's next
ForgeAgent is a strong local/reference implementation today. The natural next step is a deployed multi-tenant control plane with managed Postgres, OAuth/OIDC, browser-based approval queues, hardened worker infrastructure, and a public marketplace for proof-backed capability packages.
Log in or sign up for Devpost to join the conversation.