Mentra

Mentra isn't a regular AI tutor. It traces your gap back to concepts underneath, builds the lesson and quiz to close it, makes you teach it back, and notes + answers inside the lectures you watch.

Comment

Inspiration

Every student we know learns from YouTube lectures, and almost none of it sticks. You watch an hour of calculus, nod along, and a week later you can't reproduce a single step. The tools meant to help don't: flashcard apps make you write the cards, and AI tutors answer exactly the question you asked.

That last part is the real gap. When you're stuck on derivatives, the honest reason usually isn't derivatives. It's that limits never landed, three concepts back. A chatbot will happily re-explain derivatives forever. It has no idea what you already know, because it has no model of you.

We wanted to build the opposite: a tutor that starts by figuring out what you're actually missing.

What it does

Mentra keeps a mastery graph, a live model of every concept you've touched and how well you hold it. Ask for a topic you aren't ready for and it traces backwards through prerequisites and teaches the gap first.

Everything keys off that graph:

  • Lessons and quizzes generated for the concept you actually need, with an escalation ladder when you miss: analogy → diagram → video timestamp
  • Learn by teaching. Protégé Mode puts you on the other side, explaining the concept to an AI student that asks the naive questions. Producing an explanation is a much harder test than recognizing an answer.
  • Spaced repetition with FSRS scheduling, plus exam-date-aware planning that projects retrievability and tells you what to review and when
  • Watch & Note, a Chrome extension that rides along in YouTube lectures: capture a note at the current second with a real screenshot of the frame, ask questions about the video and get answers cited to the timestamp, and turn your notes into flashcards
  • Snap a problem and get the specific step where your work went wrong, not just the right answer
  • Peer struggle feed and study squads that group people stuck on the same concept
  • PDF export where every timestamp is a real link back to the second of the lecture it came from

How we built it

Backend: FastAPI with a LangGraph state machine orchestrating the agent nodes, Celery + Redis for the generation pipeline, and Postgres with pgvector for semantic search.

Embeddings: BAAI/bge-small-en-v1.5 through fastembed, running locally on ONNX. No API key, no PyTorch, 384 dimensions. Shared as one installable package so the backend, the seed script, and the vector MCP servers can't drift apart.

MCP servers: Four real MCP servers for video transcript search, tutor matching, calendar booking with genuine 15-minute holds, and geo search. They speak the actual MCP protocol over streamable HTTP and are queried by the orchestrator as a real MCP client. No simulated tool calls.

Extension: Manifest V3, TypeScript, shadow-DOM panel injected into YouTube's rail. Two things forced it to be an extension rather than a web page: a page can't screenshot the YouTube player (cross-origin iframe taints the canvas and toDataURL() throws), and server-side transcript fetching gets IP-blocked from cloud hosts. A content script runs on youtube.com, so both are same-origin and simply work.

Frontend: React + Vite. Deployed on Fly.io, Neon Postgres, and Vercel.

Challenges we ran into

Our embeddings were fake and we didn't notice. A placeholder hashed text into a random unit vector. It exercised the pgvector plumbing perfectly while making every ranking meaningless. Measured, "limits" scored −0.043 against "Introduction to limits | Khan Academy": a topic scoring negatively against its own matching video. With a real model, the same pair scores +0.813. Nothing errored; the search was just quietly wrong.

The extension was reading the wrong video's transcript. We read captions from ytInitialPlayerResponse, which YouTube writes once, on first page load. YouTube is a single-page app, so navigating search → video → another video leaves that global holding the previous video's data. It wasn't failing loudly. It was ingesting the wrong lecture's transcript under the new video's ID and poisoning search results. Reading from the live player API fixed it.

Deploying surfaced three silent failures in a row. The Docker image built green and crash-looped because the shared embeddings package was never copied in. Neon's connection URL carries sslmode, which SQLAlchemy's asyncpg driver rejects as an unexpected keyword. And Celery refuses TLS Redis URLs without an explicit certificate setting. But because the worker starts in the background, that error scrolls past and the app looks perfectly healthy while every queued lesson goes unprocessed.

We sized the machine wrong. Estimated 1GB from local measurements; the first real deploy OOM-killed the API during boot, because three separate processes each load their own copy of the embedding model.

The pattern across all of it: the dangerous bugs never threw an exception.

Accomplishments that we're proud of

It's genuinely deployed and genuinely works, not a localhost demo. Real embeddings, real MCP protocol servers, real spaced-repetition scheduling.

We refused the convenient fakes. The placeholder embeddings would have passed every "returns 384 floats" test forever. We replaced them and wrote tests that assert rankings are semantically ordered, so the regression can't silently return.

The extension does things a web app cannot. Frame capture and reliable transcript access, rather than approximating them.

We checked our own design work. Every color carrying text was measured for contrast; our first pastel palette failed at 3.1:1 and we darkened it until it passed. Pretty and unreadable is still broken.

78 passing tests and CI on every push, covering caption parsing, embedding ordering, topic matching, and the notes API.

What we learned

The biggest lesson was about working with AI coding agents effectively, and it came down to something unglamorous: markdown files.

We started by writing specs in .md before writing code. What a feature should do, what the edge cases were, what "done" meant. That turned out to matter far more than we expected. An AI agent with a clear written spec produces something close to what you wanted; the same agent with a vague prompt produces something plausible that you then have to reverse-engineer. The markdown file is the interface.

Three things that made the difference:

  1. Write the "why" down, not the "what." Code says what it does. A comment explaining why a threshold is 0.62, or why a note is left unmapped rather than guessed, is what stops the next person, human or AI, from "fixing" it into a bug.

  2. A README that states what's real. Being explicit about which parts were genuine and which were placeholders is exactly how we caught the fake embeddings. Documentation that flatters the project hides its problems.

  3. Never trust a green checkmark. Our worst bugs all passed their tests. The wrong-transcript bug was only found by loading the extension on a real lecture and looking. AI helps you write code fast; it doesn't relieve you of running the thing.

What's next for Mentra

Mind maps that show how it all connects. The extension will draw a live mind map of the lecture as you watch, with concepts as nodes and relationships as edges. It will then trace backwards to the prerequisites underneath and highlight where it touches topics you're already struggling with. Not a pretty diagram: a picture of how this lecture plugs into the rest of what you know, and where the holes are.

A peer help space. Post what you're stuck on in any subject and get help from students who've been there. Two things make it different: we can match you with someone who was stuck on this exact concept recently and got past it, because the mastery graph knows. Helping someone is itself a retrieval exercise, so we'll credit it back to the helper's own mastery. Anonymous by default, because the people who most need to ask are the ones least likely to raise a hand.

Quizzes that get stranger and smarter:

  • Prerequisite-targeted questions. You asked about derivatives, we quiz the limit underneath.
  • Misconception distractors. Wrong answers built from the specific error you made last time, not random noise.
  • Explain-it-back grading, scoring the explanation you produce rather than the option you picked
  • Confidence-weighted scoring. Confidently wrong is the most valuable signal in learning, and most quizzes throw it away.
  • Interleaved review that mixes topics on purpose, because blocked practice feels better and works worse
  • Predict-the-next-step problems, where you choose the move rather than check the answer

Also coming: shared squad mind maps for group study, a lecture-to-practice-set generator that turns any video into a problem set, and "what decayed since last time", a session that opens with exactly the concepts your retrievability curve says are slipping.

Built With

Share this project:

Updates

Submission history