codify

Inspiration

We have all struggled to stay consistent with LeetCode. A solved count goes up by one, and that is the only feedback you get. It does not say whether the solution was any good, which topics you are weak in, or why you should come back tomorrow. We wanted practice to feel like progress you can see.

What it does

Codify turns each LeetCode attempt into a graded, ranked result.

Submit an attempt: the problem, your code, how many test cases passed, and optionally your time and hints used.
AI-assessed feedback: AI reviews the code and returns the approaches used, time and space complexity, line-by-line issues, a counterexample when the code looks wrong, and alternative solutions.
Grades: letter grades from A+ to F- for quality, correctness, time and hints, plus an overall grade.
Skill ranks: points go to the 17 skills your code actually used. Each skill, and your overall profile, ranks from Bronze III to Diamond I.
Reasons to come back: 98 tiered badges, a streak, points-over-time charts, and a recommended next problem weighted toward your weak topics.
Import: bring in your LeetCode history by username.
Ask questions: a chat on every submission, with your code and the feedback as context.

How we built it

Stack: Next.js and TypeScript, Supabase (Postgres, auth, storage), Tailwind and shadcn/ui, Recharts, and the Anthropic API.
The LLM judges, code scores: Claude returns structured JSON and never returns points. A pure TypeScript module does all the math.
Scoring: every problem has a maximum number of points, based on its difficulty and how few people get it accepted. How much of that you earn depends on how well you solved it: code quality, correctness, time taken and hints used. Re-solving a problem you already nailed earns very little, so points come from learning something new.
Elo-style skills: each skill is rated against the problem's difficulty, like a chess rating. Solving something hard for your level earns a lot, solving something easy earns almost nothing, and failing an easy problem costs you points.
Skill decay: unpracticed skills fade on a power-law forgetting curve, fitted to published skill-decay studies. Stronger skills fade more slowly, and the largest decay happens in the first few weeks.
Process: one design document was the source of truth. We wrote test vectors before code and worked with AI coding agents against that spec.

Challenges we ran into

Rate limits: the LeetCode wrapper API we started with allows 120 requests per hour. We moved to LeetCode's GraphQL endpoint and store problems in our own database ahead of time.
Official solutions: editorials embed their code as iframes, so we fetch each approach's code separately and stitch it into the text the LLM reads.
Floating point: a perfect attempt computed to 0.9999999999999999, so "perfect" is defined by four A+ grades.
Fair imports: a problem with several tags was counted once per tag, which inflated imported ranks. We scale each tag so every imported problem counts once.
Unverified input: we never run submitted code, so results are labelled AI-assessed and the score relies on what the user reports.

Accomplishments that we're proud of

A scoring model with real reasoning behind it: Elo-style skill ratings, mastery, and decay fitted to research.
360 unit tests on the scoring engine, written from the spec's test vectors.
An import that reads a LeetCode profile in one request and never double-counts problems solved in the app.
Feedback that is useful on failed attempts too: a 0-passed submission still gets a counterexample and alternative solutions.

What we learned

An LLM works best as a narrow judge with a fixed rubric. Keeping points in deterministic code made the results consistent and testable.
Check an API's rate limits and response shapes before designing around it.
Writing the spec and test vectors first let us change the design four times in 24 hours without breaking the scoring.