Inspiration
In most schools, tests are still written on paper. A subject teacher with five sections can bring home hundreds of handwritten pages after a single unit test, and mark them at night, by hand.
Two things get lost in that pile:
- Feedback. By sheet 100, a student gets a tick and a number, not an explanation. They learn their mark, not their mistake.
- The student behind the answer. One of our sample sheets ends with "I forgot the formula again :( maths is too hard for me." At sheet 94, at 10 pm, nobody notices that.
The pile holds the most useful information a teacher has: what the class misunderstood, and who is struggling. Nobody has time to collect it. Auto-graders for typed answers don't help, because the answers are on paper.
We wanted a tool that gives teachers their evenings back without taking away their judgement.
What it does
Photograph the answer sheets. Redpen:
- Turns your answer key into a rubric. You paste it in plain text, and Redpen turns it into precise, checkable marking points.
- Reads the handwriting word for word, mistakes included, because you can't grade an answer that has been auto-corrected. Crossed-out words are left out, unreadable words become
[?], and every answer gets a legibility rating. - Grades every rubric point with evidence. Each point gets ✓ / ½ / ✗, the student's exact words that earned or lost the mark, a reason, and the misconception the answer reveals.
- Marks the working, not just the answer. For maths and other multi-step answers, each line of working is checked, and a slip loses only that step.
- Hands the teacher what it can't read. ✋ An unreadable answer isn't guessed at. It goes to the teacher to mark from the photo.
- Notices how students feel. It reads sentiment from what students actually write, such as "I am not sure about this one", and flags upset students so the teacher can have a word with them.
- Gives class insights. It shows shared misconceptions, how students approached each question, how they felt, who needs support, and a 3-step reteach plan with 10-minute classroom activities.
- Writes feedback slips for parents, translatable into Hindi, Tamil, Bengali, Marathi, Spanish or French.
The teacher can change any mark with one click. A Model ledger shows every model call, with its tokens, time and cost.
Step marking, by example
Arjun was asked for the selling price of a Rs 2400 bicycle sold at 15% profit. He wrote:
$$\text{profit} = \tfrac{15}{100} \times 2400 = 340 \qquad \text{SP} = 2400 + 340 = 2740$$
The correct profit is $360$, so a strict grader gives him $0/4$.
Redpen checks the working line by line:
- He set up the profit correctly, so he keeps that mark ✓.
- He got $340$ instead of $360$ ✗.
- He correctly added his profit to the cost price ✓. This is called error carried forward.
- His final answer was wrong ✗.
He gets 2/4, as a fair teacher would give. A student who used a different valid method, $\tfrac{115}{100} \times 2400 = 2760$, gets full marks.
How we built it
Redpen is a FastAPI backend with a no-build, single-page frontend. Every AI step runs on NVIDIA Nemotron models served by Nebius Token Factory, through its OpenAI-compatible API. Each model has one job:
| Model | Job | Why |
|---|---|---|
| Nemotron 3.5 Super VL | Handwriting OCR | It reads all of a student's pages in one call and returns structured JSON per question: the answer matched to the question number, word-for-word text, [?] for unreadable words, a legibility rating and diagram descriptions. Reasoning is off, because this is perception. |
| Nemotron 3 Ultra | Grading, step marking, approach, sentiment, class insights and translation | Grading is where a mistake costs a student marks, so it gets the strongest reasoning model with thinking turned on. Translation runs with thinking off. |
| Nemotron 3.5 Lightning | Answer key → rubric | It's fast and cheap (≈ $0.0002 per call), so the credits go to grading. |
Reasoning is switched on or off per call with chat_template_kwargs.enable_thinking, and every call uses JSON mode.
Our core design rule: models judge, code counts. The models only make judgements. Python then:
- clamps every mark to the rubric and rounds it to half marks;
- computes every total and class statistic;
- drops any "evidence" quote that doesn't actually appear in the student's transcription, and does the same for sentiment quotes;
- decides which answers go to the teacher: unreadable answers, right methods that lost marks, low-confidence grades and upset students.
A model can never mis-add a student's marks or quietly mark an answer it couldn't read.
Measuring it
We built an evaluation harness with two sets of the same 8 students' answers:
- a neat set;
- a messy set, photographed badly: cramped letters, scribbles, blur, shadow and skew, with one sheet made near-illegible on purpose.
Reading accuracy is measured at the word level:
$$\text{word accuracy} = 1 - \frac{\text{word edit distance}(\text{Redpen's reading},\ \text{true text})}{\text{words in true text}}$$
| Metric | Neat (96 answers) | Messy (64 answers) |
|---|---|---|
| Handwriting reading accuracy | 99.2% | 90.9% |
| Exact agreement with reference marks | 83.3% | 82.8% |
| Within ½ mark of reference | 100% | 96.9% |
| Answers off by more than ½ mark | 0 | 2, both already flagged for the teacher |
Grading costs about 1–2 US cents per sheet, including class insights.
We also wrote 34 offline unit tests for the marking rules. They cover:
- clamping marks and dropping invented evidence;
- step marking;
- teacher hand-off;
- sentiment;
- image handling.
On top of that, we ran end-to-end tests of the full app in a browser and against the API.
Challenges we ran into
- Faithful transcription. Vision models want to "help": they fix spelling and finish sentences, which hides exactly the mistakes a teacher needs to see. Getting a verbatim transcription with crossed-out words removed took many prompt iterations.
- Near-illegible handwriting. On our deliberately unreadable sheet, the vision model sometimes wrote a clean textbook sentence instead of the student's real words (70% accuracy on that sheet). Prompting alone didn't fully fix it. So unclear answers are flagged and shown next to the photo, and answers that can't be read at all go straight to the teacher.
- Fair step marking. Our first version gave a student 1/4 for a correct alternative method ($115\%$ of the cost price). We fixed this by carrying the teacher's own step-marking notes into the rubric and teaching the grader that a valid shortcut covers the steps it replaces.
- Numbers in AI-written text. Ultra's first class report said "5/6 students" in an 8-student class. Now Python computes every statistic, and the model is told to copy numbers, never to calculate them.
- Indian languages. Nemotron 3.5 Lightning garbled Tamil and Bengali, so we moved translation to Ultra and added a check that the reply really is in the right script.
- Reasoning leaking into answers. With thinking on and a tight token limit, Lightning's unfinished reasoning appeared in the answer field. We now control reasoning explicitly for every call.
Accomplishments that we're proud of
- An end-to-end product a teacher can actually use, from photo to parent slip.
- Grading that shows its working: every mark has the student's own words as evidence.
- An AI that knows when to ask. In our tests, every answer it got wrong by more than half a mark had already been flagged for the teacher.
- Measured accuracy, cost and latency, with every disagreement listed openly in the repo.
What we learned
- Splitting the work across specialised models (vision, fast and reasoning) was cheaper and more accurate than asking one model to read and grade at once.
- Evidence changes trust. Showing the student's own words for every mark turns "trust the AI" into "check the AI in five seconds".
- Code should own the decisions that matter. Models are good judges but unreliable bookkeepers. Keeping arithmetic, statistics and the "send this to the teacher" decision in code made Redpen far more trustworthy.
- A grader should notice the student, not just the answer. One line of sentiment, "maths is too hard for me", can matter more than the mark.
Limitations (we'd rather you hear it from us)
- The evaluation sheets are synthetic. They are rendered from handwriting fonts, and the reference marks were set by us, not by an independent teacher. Real-world accuracy will be lower.
- Answers only, not drawn diagrams. Redpen describes diagrams but doesn't award marks for them yet.
- Answers in Hindi and other scripts haven't been tested.
- Redpen is meant for classroom tests with a teacher in the loop, not high-stakes exams.
What's next for Redpen
- Evaluate on real, consented, anonymised sheets, marked independently by two teachers.
- Add a second, independent reading pass, and flag answers where the two readings disagree.
- Award marks for diagrams (labelled cells, circuits, graphs), and grade answers written in Hindi and other scripts.
- Scan a whole class as one PDF, split automatically by student.
- Track each student's misconceptions and wellbeing across tests over the year.
Log in or sign up for Devpost to join the conversation.