Diagnos

It doesn't correct your answer. It corrects your reasoning.

Comment

Inspiration

Every math app we tested said the same thing when a student failed: "Incorrect. The answer was 5."

That sentence teaches nothing. It closes a loop that should have opened one. The student now knows a number they didn't know before, and knows nothing about the thought that led them somewhere else.

Because behind a repeated algebra mistake there is almost never a typo. There is a misconception: a false rule the student genuinely believes is true, applies consistently, and has no reason to doubt. A student who writes 3x + 5 = 20 and then 3x = 20 + 5 is not being careless. They are being systematic. They believe a term keeps its sign when it crosses the equals sign, and they will keep believing it — in every exercise, on every exam, for the rest of the school year — until somebody names that belief out loud.

Naming it is the entire job. And it is precisely the job nobody has time to do. Teachers know this better than anyone; they also have thirty-five notebooks to read and forty-five minutes to read them in. Reconstructing the thinking behind each wrong answer, one student at a time, is not a human-scale task. It is not that teachers don't want to diagnose reasoning. It is that arithmetic makes it impossible.

So we stopped building an answer checker and built a reasoning diagnostician instead — one designed from the first line of code to scale from one student to a full classroom, because helping one student was never the actual problem.

What it does

The core loop lives at /analizar and has four movements. The design constraint that shaped all of them: the app must never hand over the answer.

1 · Show your work — on paper, if that's what you have

The student photographs their notebook or types their steps directly. A vision model reads the handwriting and reconstructs the procedure exactly as it was written, preserving the student's own notation rather than normalizing it into something cleaner.

This was a deliberate accessibility decision, not a technical flourish. Requiring students to retype their work in a structured editor would have excluded exactly the classrooms that need this most — the ones where the tool available is a notebook, a pencil, and a shared phone. Diagnos was built around paper first.

2 · Find the step where the reasoning broke

Diagnos ignores the final answer entirely and walks the reasoning chain looking for the first step where the logic fails.

Every step after that one is marked as carried forward, not as a new error — because it isn't one. A student who makes a single sign mistake and then executes four flawless steps on a now-wrong equation has demonstrated four steps of correct algebra. Marking those as "wrong" is not just unhelpful; it is factually false, and it teaches the student that their reasoning is unreliable when the truth is that one specific belief is.

3 · Name the misconception

The error is then classified against a fixed catalogue of twelve documented algebraic misconceptions, each grounded in published research in mathematics education: Kieran (1981), Küchemann (1981), Matz (1982), Fischbein et al. (1985), De Bock et al. (2002), Vlassis (2004), and Ni & Zhou (2005).

This is the difference between a diagnosis and a guess. The catalogue is closed and its codes are stable, which means the classification is reproducible, auditable, and — critically — aggregable. It also means the labels aren't invented by a language model on the spot: they're categories the research literature already established and named. The full catalogue, with its citations, is public inside the app at /ayuda rather than buried in our source code.

4 · Socratic dialogue — questions only, never the answer

The tutor is hard-constrained at the prompt level: it may only ask questions. No rules. No corrected step. No solution, no matter how directly the student asks for it or how long the exchange runs.

It keeps asking until the student states their own error out loud. When that happens, the model emits a sentinel token the application listens for, so the app knows the student genuinely arrived at the insight rather than guessed a number that happened to be right. Only then does the session advance.

The pedagogy here is not decoration. A misconception that is told to a student competes with a belief they already hold and usually loses. A misconception the student articulates themselves replaces it.

5 · Targeted practice — three exercises against your error

Diagnos then generates three brand-new exercises aimed at that specific misconception, escalating in difficulty. The third is built so the false rule produces an obviously absurd result — so the exercise itself refutes the belief without anyone having to assert anything.

And here is the judgment we're most proud of: practice is graded on two independent axes. Was the answer correct? and did the student repeat the misconception? — evaluated separately, on purpose.

A correct answer with the flawed reasoning still visible does not count as overcome. An arithmetic slip with correct algebraic reasoning does. Collapsing those two axes into a single right/wrong score is exactly the mistake every other tool in this category makes, and keeping them apart forces the model to reason about student intent rather than about a number.

6 · The teacher heat map — where this stops being a tutor

Because the catalogue codes are stable, individual diagnoses aggregate into something a teacher can act on. At /docente, a teacher sees which false ideas dominate their classroom this week, ranked, with a second layer showing how much of it has already been overcome.

This is the piece that scales the product from helping a student to helping a class of thirty-five — which is the problem a math teacher actually has every single day. Instead of guessing what to reteach on Monday, they can see it. And instead of reteaching a rule to everyone, they can open class with a counterexample of the dominant misconception and let the room dismantle it out loud.

Beyond the core loop

Diagnos shipped as a complete product, not a single demo flow:

  • /tutor — PDF study companion. Upload course material and get an elaborated summary, key points, flashcards, and exercises. Built for the student whose real bottleneck is a fifty-page handout, not a single equation.
  • /grafica — function laboratory. Trigonometric, rational and other functions, with correct asymptote detection (which is genuinely easy to get wrong, and which we cover with automated tests).
  • /ayuda — the full misconception catalogue, with academic references, readable by students and teachers alike. We publish the pedagogy rather than hiding it.
  • Full accounts. Sign-up, sign-in, email password recovery, verified role changes, sign-out — all working against a real auth backend.
  • Guest mode. No account required to try the core loop. A judge can be diagnosing a real exercise fifteen seconds after opening the link.
  • Gamification with a purpose. Streaks with milestones, accumulated points, achievement stickers — designed to bring a student back tomorrow, not just to look good during a demo.
  • Genuinely bilingual (EN/ES) — including the AI-generated output, not just the interface chrome.
  • Plus /perfil, /contacto, /terminos and /privacidad: the unglamorous pages that separate a deployed product from a prototype.

How we built it

Frontend

Next.js 14 (App Router, JavaScript), Tailwind CSS with a custom dark design system, and a hand-rolled i18n layer built on React context with synchronized EN/ES dictionaries. We wrote our own internationalization rather than pulling in a library because the hard part wasn't the interface strings — it was keeping model-generated content in the user's language consistently, which no i18n package solves for you.

Backend

Appwrite for Auth and TablesDB (diagnoses, overcome-records, messages), unpdf for server-side PDF text extraction, deployed on Vercel.

Inference — Groq, with model selection per task

We deliberately avoided the "one model for everything" default. Each task has different constraints, so each task got a different model:

Task Model Why this one
Diagnosis from a photo qwen/qwen3.6-27b The only Groq model with image input
Diagnosis, Socratic dialogue, practice openai/gpt-oss-120b The only one with strict json_schema — non-negotiable when the UI renders structured steps
PDF tutor llama-3.3-70b-versatile Highest tokens/minute ceiling, with no reasoning overhead on a task that doesn't need it

The actual engineering is in the prompts

The plumbing is standard; the difficulty was elsewhere. Getting a model to reliably withhold an answer, to distinguish a carried-forward consequence from a fresh error, and to judge reasoning independently of results — those took iteration with explicit positive and negative examples baked into each system prompt: this is what a good response looks like, and this is what a forbidden one looks like.

The dual-axis practice judgment in particular is not a prompt you write once. It required the model to hold two separate questions in mind about the same student answer and resist the overwhelming pull to let a correct number settle both of them.

Security and privacy

We treated this as a product that would handle minors' schoolwork, because that is what it is.

Anonymity is structural, not promised. Diagnoses are stored with no user identifier at all — only a misconception code and a classroom code. The heat map works precisely because it never needed to know who made the mistake. There is no "we don't look at it" policy to trust here; the association simply doesn't exist in the database.

Zero client permissions on data tables. Every sensitive read and write goes through a server route that verifies identity by JWT — never by what the client claims to be. A student's classroom is derived from the verified token, so nobody can pull another class's data by editing a parameter in a request.

The teacher role is not self-service. Promotion to teacher requires a code the server verifies against a secret (CODIGO_DOCENTE), using the user's JWT to prove who they are. This closed a real hole we shipped in an early version — see below.

Spam hardening. The contact form carries a hidden honeypot and length limits against automated submissions.

Challenges we ran into

Making the tutor shut up

Language models desperately want to be helpful, and "helpful" defaults to giving the answer. Our first Socratic tutor caved almost immediately: two rounds of student frustration and it would volunteer the rule, the corrected step, and the solution, all in one enthusiastic paragraph.

Fixing it required explicit negative examples in the prompt, a strict one-question-per-turn rule, a defined escalation path for a stuck student (a minimal numeric counterexample — still not the answer), and the [DISCOVERED] sentinel so the application could distinguish genuine insight from a lucky guess. The tutor is now, by design, the least immediately gratifying part of the product — and the part that actually teaches.

A privilege escalation we shipped and then closed

Our first version let any student flip themselves to teacher with a single click, which would have exposed classroom-level aggregate data to anyone curious enough to try. We found it, and we closed it properly: role promotion now requires a server-verified code, and identity is proven by JWT rather than asserted by the client. That fix cascaded into the broader decision to remove client permissions from data tables entirely.

We're including this rather than quietly omitting it, because how a team responds to finding a hole in their own product says more than a clean-looking feature list.

Rate limits under real classroom load

We ran a local load test simulating an entire class — 35 simultaneous requests — and Groq's free tier does not survive it. Most clients receive a 429.

Rather than hide that, we engineered around it honestly: all four AI endpoints return a clear "wait a minute" message instead of a generic failure, and we disabled the SDK's automatic retries so that error surfaces in seconds rather than after a minute of silence. That timing detail matters more than it sounds — Vercel functions are cut off at 60 seconds, so a silently retrying SDK turns a recoverable rate limit into an unexplained timeout.

This is a free-tier capacity limit, not a bug, and the system now communicates it clearly instead of failing confusingly.

Keeping two languages honest

Translating an interface is straightforward. Keeping generated content — diagnoses, Socratic questions, generated exercises — consistently in the student's language, while the underlying prompts and the misconception catalogue stay stable, took real care across every AI endpoint.

Quality and testing

42 automated tests running on Node's native test runner — deliberately zero new dependencies — covering PDF chunking, streak logic, misconception catalogue integrity, and asymptote detection in the function lab.

Continuous integration on GitHub Actions runs the full suite plus a production build on every push, so a change that passes tests but breaks the build never reaches main unnoticed.

The catalogue-integrity tests deserve a mention: because every diagnosis, every generated exercise, and the entire teacher heat map key off those codes, a single typo in the catalogue would silently corrupt aggregate classroom data. Testing it is cheap; discovering it in production would not have been.

Accomplishments we're proud of

  • It is deployed and genuinely working against real backends — not a mock, not a video-only prototype. Guest mode means it can be verified in fifteen seconds.
  • The misconception catalogue is defensible pedagogy, not invented labels. Every code cites its source, and the catalogue is published in-app for anyone to check.
  • The dual-axis judgment — correct ≠ overcome — is the single design decision that most separates Diagnos from every "AI homework helper," and it exists because we asked what mastery actually means rather than what is easy to score.
  • Complete authentication with verified role changes, and a privacy model where anonymity is enforced by schema rather than by policy.
  • The heat map turns a personal tutor into a classroom instrument — the leap from one student to thirty-five.
  • A real bilingual product, AI output included, from a small team on a hackathon timeline.

What we learned

The hard part of an educational AI product is not the model. It is deciding what the model is not allowed to do.

Every meaningful design decision in Diagnos was a subtraction. Don't grade the final answer. Don't reveal the rule. Don't accept a correct number as proof of understanding. Don't mark carried-forward steps as new errors. Don't store who made the mistake. The product got better every time we removed a capability that felt helpful and was actually corrosive to learning.

We also learned something from the research that reshaped an entire feature: misconceptions do not disappear when you state the correct rule. They are replaced when a student watches their own rule produce something absurd. That single finding is why the third generated exercise always drives the false belief to a visibly ridiculous conclusion — the exercise argues, so the tutor doesn't have to.

Honest limitations

We would rather state these plainly than have them discovered:

  • Diagnos is sophisticated prompt engineering on hosted models, not a fine-tuned or self-trained model. If fine-tuning is what's being rewarded, there is nothing here to show; if intelligent application of existing models to the right problem is, the project defends itself.
  • It is a functional prototype, not an efficacy study. The design is built for learning impact, but we do not yet have data from a real classroom using it over weeks. We are claiming a well-designed intervention, not a proven one.
  • Free-tier inference limits mean it is not yet ready for 35 concurrent students. We know this because we measured it rather than assumed it.

What's next for Diagnos

Widen the catalogue. Extend beyond first-year algebra into geometry and physics, keeping the same standard: every misconception cited, never invented.

Run a real pilot. Put Diagnos in an actual classroom for a full term and measure the only metric that matters — whether misconception recurrence drops. That is the study this prototype was designed to make possible.

Ship offline capture. Let students in low-connectivity schools photograph their work now and sync when they have signal. This is the reason we designed around paper, pencil and a camera in the first place, and it is the difference between a tool for well-resourced classrooms and a tool for the ones that need it most.

Live app: https://diagnos-hazel.vercel.app Source code: https://github.com/neterfk-coder/Diagnos-proyect.edu

Built With

Share this project:

Updates

Submission history