Inspiration

Every one of us has walked out of an exam "studied for" and realized we had only memorized the answers. Most study tools make that worse. Flashcards reward recall, quizzes reward guessing, and AI chatbots hand you the answer the moment you ask. None of them can tell the difference between getting something right and actually understanding it.

We wanted a tutor that refuses to confuse the two.

If you get a question wrong, it should identify the exact idea you misunderstood, not just show the correct answer. If it teaches you something, it should make you prove that understanding on a different problem. And even then, it should not call that mastery until you can retrieve the idea again later.

That became ProveIt: an AI mastery and evidence engine built around one question:

Do you actually know this?

What it does

ProveIt turns a course into a map of concepts and tracks how well a student actually understands each one.

• Test this concept starts as a conversation, not a quiz. The tutor asks what you think a concept means, responds to what you actually say, teaches step by step when needed, and never puts the answer into a tap-to-reply button.

• When you're ready, ProveIt checks your understanding. Three independent structured grading passes evaluate the visible answer. If they agree on a misconception, ProveIt names that specific gap and can quote the student's own words as evidence. If the evidence is ambiguous, it asks for clarification instead of inventing a diagnosis.

• A targeted lesson repairs only that gap. ProveIt then gives a materially different transfer question. Passing that makes the concept Provisional, not Mastered.

• Mastery requires retention. A delayed review later checks whether the student can still retrieve the idea. Only then does the concept become Mastered.

• Bring your own course. Upload notes, slides, PDFs, Word or PowerPoint files, add a curated course, or import supported notes. ProveIt builds a concept map and generates source-grounded tutoring, quizzes, practice material and mastery checks from the course content.

• Talk it through. ProveIt supports a live, interruptible tutor powered by an ElevenLabs conversational agent over WebRTC. The voice tutor receives the same concept context as the text tutor, and the transcript returns to the same Tutor conversation.

• ASL Assistant adds sign input directly to the Tutor. Students can fingerspell answers using American Sign Language letters. A local Python service uses MediaPipe hand landmarks and an ASL classifier, waits for a stable prediction, and places the letters into the message box for the student to review before sending.

• Rank is earned from evidence. Topic point allocations are weighted by importance in the course map. Quiz and explanation performance can earn their respective allocations, but repeated questions cannot be farmed for points. Rank and mastery are deliberately separate: points do not replace transfer or delayed retrieval.

• Accessibility is part of the product rather than a separate page. ProveIt supports dyslexia-friendly fonts, letter/word/line spacing, line focus, read aloud, text size, zoom, dark mode, voice interaction and sign input.

How we built it

• Frontend: Next.js 16 App Router, React 19, TypeScript and Tailwind CSS, with responsive layouts, dark mode and a frosted-glass interface.

• AI: The Vercel AI SDK connects ProveIt to Gemini through OpenRouter, with optional direct Gemini support. We use the model for tutoring, generation and structured evaluation, but application code — not the model — owns mastery state transitions.

• Learning engine: Every concept is represented by a canonical, versioned ConceptLearningSpec containing the assessment question, rubric, known misconceptions, clarification probe, targeted remediation, transfer question, delayed-review question and source references.

• Evaluation: Assessment, transfer and review use three independent structured model evaluations and majority agreement. A diagnosis may only select from declared misconception IDs, and evidence quotes must come from the learner's current answer. Ambiguous evidence stays uncertain instead of being forced into a diagnosis.

• Grounding: Uploaded course material is split into exact server-selected excerpts. Generated learning content may reference only server-created source IDs. Each generated version is immutable, so changing course material cannot silently rewrite the evidence used for an earlier assessment.

• Mastery: ProveIt tracks not started, developing, needs work, provisional and mastered. Transfer establishes Provisional; delayed retrieval establishes Mastered. One immediate correct answer never does.

• Course pipeline: PDF and text extraction happen locally when possible using unpdf. Model calls are reserved for tasks such as scanned/image content, course structuring and generated learning material. Larger generation jobs are parallelized to keep the experience responsive.

• Voice: ElevenLabs powers the live conversational Tutor over WebRTC. Our server keeps the API key private, issues a short-lived conversation token, and provides the agent with a concept-specific tutor brief and the current conversation so the spoken experience continues from the same context.

• ASL: Browser camera frames are sent to a local Python inference service. MediaPipe extracts hand landmarks, a pretrained ASL classifier predicts letters, and stability/confidence filtering prevents noisy detections from being inserted into the student's answer.

• Data and accounts: SQLite persists learner state locally for the hackathon build. Clerk provides optional authentication, while ProveIt's server-side user boundary and AsyncLocalStorage isolate course state, evidence, materials, rank, calendar data and generated learning content per user.

• Ranking: Course-map importance determines frozen per-topic allocations. A standard 20-topic course contains 3,000 quiz points, 7,000 explanation points and a 2,000 completion bonus. 70% is Proficient with no points; 90% is Assessment Mastered and can earn the relevant allocation. Distinct-question fingerprints prevent repeated quiz farming.

Challenges we ran into

• Speed. Our first course-building pipeline sent far too much PDF content through the model and could take minutes. We moved ordinary PDF/text extraction local, parallelized model work, and reduced generation to the places where AI was actually useful. Uploaded courses can pre-generate learning content in the background, while built-in concepts generate on demand so simply opening a course does not cause a burst of model requests.

• Not guessing. It is easy to make an AI grader sound confident. Making it say "I need more evidence" when the student's answer is genuinely ambiguous took deliberate evaluator design, constrained misconception IDs and extensive test conversations.

• Not giving answers away. Early quick-reply buttons could accidentally suggest the answer to the Tutor's own question. We restricted those buttons to clarification choices and actions such as asking to be taught or requesting an assessment.

• Making rank honest. Points come only from scored assessment evidence, each allocation pays once, repeated questions cannot be farmed, resetting or deleting and re-adding a course cannot regenerate rewards, and mastery still requires transfer plus delayed retention.

• Making four people's work behave like one product. Backend, frontend, course generation, voice and ASL evolved in parallel. We spent significant time integrating those systems, resolving conflicts, adding cross-system regression tests and repeatedly hardening the full application as new teammate changes landed.

Accomplishments that we're proud of

• A complete learning loop that goes from a student's answer to a specific misconception, a targeted repair, a materially different transfer question and delayed proof of retention.

• A Tutor that teaches by asking instead of immediately telling, while preserving the same learning context across text, voice and sign input.

• A course system where uploaded material can become a concept map, Tutor context, quizzes, practice papers, notes and mastery assessments.

• An evidence model that preserves exactly what the learner answered, what source/version was used to judge it, and how mastery changed.

• A rank system tied to meaningful assessment evidence instead of clicks, streak farming or repeated easy questions.

• 578 automated tests across 81 test files, plus repeated repo-wide integration and hardening passes across the mastery engine, AI generation, ranking, accounts, course flows, ElevenLabs voice and ASL integration.

What we learned

• Grading understanding is much harder than grading answers. Some of the most useful evidence is what a student almost says, which is why an uncertain verdict can be more correct than forcing a right/wrong decision.

• The AI should not own the product's truth. Models are excellent at generating and evaluating bounded artifacts, but deterministic application logic should decide state transitions, reward accounting and what constitutes mastery.

• Grounding needs provenance, not just a prompt. Saving exact source excerpts and immutable learning-spec revisions made it possible to trace why an assessment existed and prevent later content changes from rewriting old evidence.

• Most perceived AI latency was actually engineering latency. Local extraction, parallel generation, caching and avoiding unnecessary work improved the experience more than simply changing models.

• Accessibility is more useful when it is embedded into the core learning experience. Voice, sign input, readable typography and read-aloud tools matter more when students can use them inside the same Tutor instead of being sent to a separate accessibility feature.

What's next for ProveIt

• Hosted, durable storage so ProveIt can move beyond a single-machine hackathon deployment.

• Stronger motion-based ASL recognition and eventually full signed-word/sentence understanding rather than only fingerspelled letters.

• End-to-end imports from services such as Granola and Wispr Flow using the connector infrastructure we already built.

• Calendar write-back so ProveIt can automatically schedule recommended study and delayed-review sessions.

• LMS integrations so upcoming assessments, course material and deadlines can automatically become part of a student's mastery map.

• More long-term learning analytics around retention, misconception patterns and what types of remediation work best for each learner.

Built With

Next.js, React, TypeScript, Tailwind CSS, Vercel AI SDK, OpenRouter, Google Gemini, ElevenLabs, Clerk, SQLite, Python, MediaPipe, MCP, Vitest, Testing Library, unpdf, Zod

Built With

• next.js
• react.js
• typescript