-
Forge Thumbnail
-
Netlogs Report
-
Audit
-
WebUI
Track 1
Inspiration
Agent fleets can complete one-off tasks, but their useful experience usually disappears into transcripts. A worker may discover the right API sequence, recover from a failure, or find a reliablerepair strategy—but the next run often starts from zero.
Forge was inspired by a simple question:
│ What would it take for an agent to learn from its work without treating every model response as truth?
The goal was to build a learning layer around agent execution: one that records what happened, verifies the result independently, and only promotes a lesson after it survives evidence and evaluation.
What I Built
Forge is a local-first control plane for agent fleets. It connects a task goal to a bounded workflow:
─ text goal → context and task contract → failing baseline → bounded repair → independent verification → structured reflection → candidate skill → promotion gates
The submission MVP demonstrates a real, deterministic slice of this workflow. A frozen C0 task starts with a failing implementation. Forge records the baseline failure, applies one bounded repair,runs the verifier again, and produces a structured candidate skill from the observed evidence.
The verifier owns the final pass decision. Reflection can propose a lesson, but it cannot promote itself. Run events are persisted in SQLite and JSONL, with the configured reflection path also instrumented through Neatlogs.
Forge is designed to work alongside Agent Orchestrator. AO owns the worker and worktree execution layer, while Forge owns the evidence ledger, verification policy, repair boundaries, learningartifacts, and promotion rules.
The biggest lesson was that memory is not automatically learning.
A transcript, a successful-looking final message, or an idle worker session is not enough to prove that an agent accomplished something. A useful lesson needs:
• a recorded failure and repair history; • an independently verified outcome; • provenance linking the lesson to the run that produced it; • applicability and regression checks; • evidence that the lesson improves a later run.
We also learned that honest boundaries are part of the product. The current artifact proves the bounded local failure → repair → verification → reflection path. It does not yet prove autonomous AOcompletion, cross-harness transfer, or a generally validated skill. Forge reports those gaps instead of filling them with agent narration.
How I Built It
We built Forge as a Python CLI with a local SQLite ledger and JSONL event journal. Its main components are:
• Planner and context builder for creating bounded task contracts; • Verifier for running acceptance checks independently of worker narration; • Repair loop limited to one bounded attempt; • Reflection layer for producing schema-valid candidate skills; • Evidence ledger for preserving goals, failures, repairs, candidates, and verdicts; • Promotion boundary that keeps candidate skills separate from validated skills; • AO integration boundary for worker sessions and worktrees; • Neatlogs instrumentation for the configured submission and reflection workflow.
The final observed MVP run preserved both sides of the story: the baseline failed, the repaired version passed, and the resulting skill remained explicitly marked as candidate.
Accomplishments I'm proud of
I’m proud that Forge is built around proving what happened, not just showing a successful-looking agent demo.
We built a real bounded workflow where a frozen task starts with a failure, receives one repair attempt, passes independent verification, and produces a traceable candidate skill. The run is preservedin a SQLite/JSONL evidence ledger, and the reflection step cannot promote its own output.
I’m also proud of the honesty in the project. Our AO worker attempts did not produce the requested artifact, so we did not call them successful. That failure shaped one of Forge’s main rules: ahealthy daemon, an idle session, or a confident model response is not proof of completion.
Along the way, we added the verifier, repair boundary, structured reflection, candidate-skill format, Neatlogs instrumentation, and a local dashboard/CLI around the workflow. The result is not a claimthat agents can already learn everything. It is a working foundation for measuring whether they actually do.
Challenges
The hardest challenge was avoiding false confidence.
It was tempting to treat AO readiness, a healthy daemon, or a listed session as proof that an autonomous worker had completed the task. We found that these are only lifecycle observations. The workerstill needs to produce the requested artifact, and that artifact needs independent verification.
Another challenge was separating reflection from authority. A model can describe a failure convincingly and generate valid JSON, but that does not mean the proposed skill is correct or broadly useful.Forge therefore treats reflection as a proposal source, while the verifier and future promotion gates remain authoritative.
We also had to keep the evidence scope precise. The Neatlogs readback verifies the instrumented Forge submission/reflection path, not every AO worker or fleet span. Similarly, the current AO setupobserved OpenCode authorization, but not a completed cross-harness transfer.
What Comes Next
The next proof step is not a larger prompt. It is a controlled experiment:
- authorize a second execution harness;
- give a fresh worker the same bounded task and validated context contract;
- compare it against a no-skill baseline;
- run held-out no-regression checks;
- report the result with acceptance, cost, time, and intervention evidence only when observed.
Forge is built around that standard: agents may move quickly, but learning has to be earned.
