Exam Proof

A practice run for your quiz.

Comment

Inspiration

Every teacher has lived this moment. Ten minutes into the quiz, a hand goes up: question 4 has two right answers. Now the class is confused, the teacher is fixing the quiz on the fly, and the grades for that question are unfair.

Software gets tested again and again before anyone uses it. Quizzes don't. We wanted to give every quiz a practice run before real students ever see it.

What it does

Paste a quiz, or snap a photo of a printed one. Five simulated students take it without the answer key:

  • Maya (struggling)
  • Sam (average)
  • Priya (careful top student)
  • Leo (fast guesser)
  • Omar (literal reader)

A head teacher then reads the answer key and the whole class's answer sheets and flags the questions worth a second look. The flag types are:

  • two defensible answers
  • answer key likely wrong
  • answer leaked by another question
  • ambiguous
  • wording trap
  • too easy

Each flag comes with a plain-English reason, every student's answer and reasoning, and a minimal suggested rewrite. The teacher can accept it in one click or ignore it, then copy or download the revised quiz. These are flags for the teacher's review, not a verdict.

How we built it

  • NVIDIA Nemotron 3 Nano ×5, in parallel, thinking off: the students. Each persona is a system prompt, and the server removes the answer key before these calls. With thinking off they answer on instinct, which is both more like real students and about 3× faster.
  • NVIDIA Nemotron 3 Ultra ×1, reasoning on: the head teacher. It is the one heavy-reasoning call per run, and it sees the key plus all five answer sheets.
  • NVIDIA Nemotron 3.5 Super VL: turns a photo or PDF page into structured questions. The result is always shown to the teacher to confirm before the run.
  • Nebius Token Factory: every model call goes through its OpenAI-compatible API with plain fetch, no SDK.
  • One Cloudflare Worker: serves the static UI and the API. Results stream over Server-Sent Events, so each student's card flips to "done" as its call returns. Nothing is stored; the only server state is a hashed per-IP rate-limit counter in Workers KV that expires after an hour.

A 10-question run takes 12–45 seconds and 10k–20k tokens, which costs a few cents at most.

Challenges we ran into

  • Nano with thinking off copied our example too literally. Given a one-item JSON example, it sometimes answered only question 1 of a 10-question quiz. We fixed it by listing every expected question ID in the prompt, making the repair message name the missing IDs, and running the repair attempt with thinking back on.
  • Ultra treated our rules of thumb as hard rules. In one run it wrote "no student picked it, so clear" about a question with two correct answers. Telling it the rules are hints, and that its own subject knowledge decides, fixed the missed flags.
  • The model catalog was hard to navigate. Model IDs use three different naming styles, and the reasoning field is named differently per model (reasoning vs reasoning_content). We found the right IDs by calling /v1/models and handled both field names.

Accomplishments that we're proud of

  • On the built-in sample quiz, which has 3 deliberately broken questions, all 3 were flagged in 9 of 9 runs, and a clean control quiz got no false flags.
  • A full run is just six model calls, finishes in well under a minute, and costs a few cents.
  • Turning thinking off for the students made them both faster and more realistic.
  • The whole app is one Cloudflare Worker with no database and no stored quizzes, which matters for teachers' privacy.

What we learned

  • Evidence from a simulated class beats one model's opinion. Ask a single model "is this quiz OK?" and it says Q4 looks fine. A class that splits 3 to 2 on Q4, with the careful student writing "B and C are both renewable", shows the problem.
  • Small models with thinking off copy few-shot examples very literally, so the prompt has to spell out exactly what a complete answer looks like.
  • A judge model needs to be told its rules are hints. Otherwise it applies them mechanically and overrides its own subject knowledge.
  • Fast models and slow models have different jobs. Five cheap, fast calls produce the evidence; one heavy reasoning call makes sense of it.

What's next for ExamProof

  • LMS integrations (Google Classroom, Canvas)
  • Calibration against real class results (item-response theory)
  • Custom personas and class sizes
  • Multilingual quizzes
  • Saved history of quizzes and fixes

Built With

Share this project:

Updates

Submission history