Inspiration
I work with LLMs every day — somewhere between a solution architect and a deploying engineer, with a background in data science and ML. The same task on a different model, and suddenly the output drifts or hallucinates. Even with evaluation in place, I kept ending up with a quieter, deeper problem: a trust problem — what can I trust, which model, when, for which task? Then the skill era hit, GitHub filled with thousands of skill bundles, and the trust problem didn't add up — it multiplied. Peter Steinberger's OpenClaw lit the spark: my first goal was to make ClawHub safer — not security-safe (that's NemoClaw's job), but safer in results, reproducibility, and scalability.
What it does
clawreinforce is CI for agent skills. Point it at a skill from anywhere and it: scans it as an untrusted dependency, certifies its declared checks with deterministic scoring, signs the result to the exact skill fingerprint (Ed25519), and gives an install/review/reject Guard verdict. Arena measures the same model without the skill and with it — the real uplift. Improve rewrites a skill in a gated loop and shows why it got better, and for which models. An LLM never judges another LLM.
How I built it
The spec was distilled from a two-month predecessor and a first Build Week rebuild. V2 is a clean-room rebuild, implemented entirely in Codex sessions from public contracts, frozen examples, and behavior tests — no V1 code or git history copied. GPT-5.6 isn't just what built it; it ships inside the product as a certification tier and an Arena executor. It's deliberately lean: Python 3.11, standard-library HTTP/SSE, no frontend framework, and twelve bounded MCP tools so Codex can drive it directly.
Challenges I ran into
Keeping the whole thing honest: a missing key or a crashed run must be its own n/a with a reason, never a fake 0%. Keeping scoring deterministic — the LLM proposes, code decides. And a real measurement trap: my first uplift fixture was too easy (every model scored 10/10 → it measured nothing), so I had to build a harder one before the experiment could answer anything. My honest open edge: I haven't fully translated the reinforcement-learning-style reward function into technical loop workflows yet — the Improve loop today is a gated rewrite, not a trained policy.
Accomplishments that Iam proud of
A verify-before-install Guard running end-to-end on a real ClawHub skill. Fingerprint-bound, offline-verifiable signed certificates. And a measured answer to a real design question — on a noisy 20-case fixture, hardening lifted a weak model from 65% to 95%, with the fewshot arm repairing every targeted case in one pass and zero regressions. It runs zero-key on a fresh clone in 60 seconds.
What Ive learned
Verify the work, not the transcript — the principle I share with SWE-bench, TAU-bench, Anthropics paper, evaluation tools like promptfoo and of course Peter's ClawBench. Rules generalize; few-shot examples are model-bound — rewritten rules transferred to a foreign model (+20 points), a mined example block didn't. And a good spec is a force multiplier: two months of lessons became a clean rebuild in days.
What's next for clawreinforce
Turning the reward function into a real training loop. Adversarial script-probing in the sandbox. Shareable task packs with a capability ladder. An MCP-native path so agents self-verify skills mid-session. UI/UX optimization for also beginners and easy understanding. And a hosted SaaS with signed-badge distribution — so anyone can verify and trust a skill, from anywhere.
Log in or sign up for Devpost to join the conversation.