LearnLedger

An AI examiner for the self-taught: defend what you learned, get a Skill Passport recruiters can actually inspect. Findings, not scores.

Comment

Inspiration

For months, I've been learning system design the way a lot of developers do now: through an AI tutor, a structured curriculum, assignments, and feedback. It works arguably better for me than any course I'd have paid for. But at the end of it, a paid platform would have handed me a certificate. My version produced detailed notes and mini exercises.

When I sat with that, the certificate stopped looking like what I was missing, because today it just attests to completion. And completion is exactly what AI made cheap. The same is true of the portfolio piece and the take-home: they prove an artifact exists, not that the person in front of you can reason about it, defend a trade-off, or notice when their own design breaks.

The thing that does prove it — an oral exam, a viva, a skeptical senior engineer asking "why this and not that?" doesn't scale and isn't available to a self-taught learner at all. So I built one: an examiner that reads what you claimed to learn, questions you on it, refuses to be charmed by fluent answers, and writes down exactly what it found. The curriculum in the demo (SKILL.md in the repo) is the real one I've been studying from — LearnLedger's first user is its builder.

The thing that does prove it, an oral exam, a viva, a skeptical senior engineer asking "why this and not that?" doesn't scale. For a self-taught learner, it usually arrives only once, in the interview itself, with the stakes already at their highest and no record of it afterwards. So I built an examiner that reads what you claimed to learn, questions you on it, and writes down exactly what it found. The curriculum in the demo (SKILL.md in the repo) is the real one I've been studying from; LearnLedger's first user is its builder.

What it does

A learner publishes a Skill, a public Markdown curriculum that states what they set out to learn. GPT-5.6 extracts a competency map from it, and every competency must cite a verbatim excerpt from the source, which the server verifies before storing. That map is the public bar the learner will be judged against.

Private study notes are added next, but notes are treated as a claim, not proof. Unrelated notes are rejected outright; accepted notes are mapped against every published competency (absent/thin/partial/substantial, each with a verified quote). The server, not the model, then picks what to test: claimed competencies first, so the claim gets verified. Four scenario questions follow, then at most two typed adaptive probes, and none if the answers were already complete; an unnecessary probe manufactures doubt.

Grading is error-first: incorrect claims and missed considerations are assessed before any strength can be recorded, so a fluent, confident, wrong answer cannot be credited. What comes out is a Skill Passport with strengths, recovered understanding, shaky areas, and what wasn't evidenced. Findings, not scores. The learner controls publication: identity, exact duration, and answer excerpts are each opt-in per passport, and nothing is public until they publish the draft.

How I built it

I built this during Build Week in collaboration with Codex, which wrote the code across the stack — React + Vite frontend, Express backend, SQLite via node:sqlite. I brought the product decisions: qualitative findings instead of scores, evidence as claim rather than proof, server-owned question targeting, consent-gated publication. GPT-5.6 runs server-side only, through structured-output calls for map extraction, evidence coverage, question planning, adaptive probes, and the final examiner pass with a constrained copy-edit pass that can improve wording but cannot add, remove, or reclassify a finding.

Challenges

The hardest work was proving the examiner deserved trust. I adversarially tested it and found three structural failures: sourdough-and-astrology notes produced a polished system-design exam (the planner never saw the evidence); a confidently wrong answer was recorded as a strength (the grader was told to quote, never to judge); and omissions were generic topic labels instead of scenario-specific reasoning gaps. Each was fixed and re-tested with the same probes — the fixes are documented in the README.

Beyond the model: cross-site cookie blocking killed my original split deployment, so I moved to same-origin serving on a VPS; and designing consent properly meant rebuilding publication as a private draft the learner reviews before anything gets a public URL.

What I learned

Prompts are claims, and claims need adversarial tests for every trust property that LearnLedger advertises exists, because a probe broke the naive version first. Verification beats instruction: the durable guarantees came from server-side checks (verbatim excerpts, server-picked targets, schema validation), not from asking the model nicely.

What's next

The engine generalizes: I tested it end-to-end on a visual-design curriculum without changing a line. Education, training, certification, anywhere learning needs to become credible, inspectable evidence.

Share this project:

Updates