Lucid AI

Everyone built an AI that teaches. We built one that listens — and measures what you actually understand.

Comment

Inspiration

A student re-reads a chapter three times, closes the book, and feels ready. The next morning the paper asks why, and everything evaporates. That gap has a name in memory research — the illusion of competence: re-reading raises your feeling of knowing while leaving actual recall flat. Fluency gets mistaken for understanding.

Meanwhile, every AI education tool built in the last two years does the same thing: it explains to you. Which is exactly the activity that creates the illusion in the first place. Watching a brilliant explanation feels like learning. It isn't.

Feynman had the inversion sixty years ago, and psychology calls it the protégé effect: the fastest way to find the edge of your understanding is to try to teach it. Nobody had built that as software, because the hard part isn't the chat — it's measuring an explanation.

So the thesis became one line:

Everyone built an AI that teaches. We built one that listens.

What it does

You pick a concept. Before you type a word, Lucid asks how confident you are (1–5), captured before any feedback so it can be checked against reality later. Then you explain the concept out loud (Web Speech API) or by typing.

From there the engine runs, and the LLM never touches your score:

  • The Grasp Score (0–100) — three independently computed dimensions:

$$\text{Grasp} = 0.5,C_{\text{coverage}} + 0.3,C_{\text{correctness}} + 0.2,C_{\text{depth}}$$

  • The understanding map lights up node by node, showing which ideas you actually covered.

  • The Parrot Detector decides whether it was your explanation or the textbook's, using two signals that must both fire.

  • A confused-student persona asks about your detected gaps — one targeted question at a time, never a lecture.

  • Two causal "why" questions close the session, because coverage without causation is just a longer list.

  • The report shows the gauge, dimension bars, parrot verdict, missed ideas, contradictions, and a confidence check: you said 5/5, you scored 58 → overconfident. That mismatch is the whole thesis.

Loop Mode converts skeptics. It scores you, generates a micro-lesson (under 150 words) covering only your gaps, hands the floor back — "now explain it back to me" — re-scores, and animates the improvement.

Verified run: 71 → 81 (+10), coverage 53% → 82%, map 3/7 → 6/7.

For teachers: one six-character join code, no accounts. The dashboard doesn't rank students — it surfaces what the class missed. An idea only becomes a class-wide weak point once two or more students miss it, because repeated mistakes mean "re-teach this," not "one student struggled."

In our four-student run, three of four never explained where the energy comes from. That isn't four students to remediate. It's one sentence to fix tomorrow.

How we built it

The engine (the actual product). TypeScript, server-side, lib/ml/:

  1. Segmentation — split explanations on strong clause boundaries; merge fragments under 4 words.

  2. Coverage — embed concept-map nodes and clauses with MiniLM-L6-v2. A node counts as covered when its best cosine similarity exceeds 0.55; node weights are normalized so $\sum_i w_i = 1$.

  3. Correctness — a DeBERTa-v3 NLI cross-encoder checks claims against reference facts: contradiction at $p > 0.8$, entailment at $p > 0.6$.

  4. Depth — cosine similarity between your "why" answers and the expected idea, mapped through $\mathrm{clamp}!\left(\frac{s - 0.2}{0.55},, 0,, 1\right)$.

  5. The Parrot Detector — fires only on a conjunction:

$$\text{parrot} = (\text{sim}_{\text{textbook}} > 0.72) ;\wedge; (\text{trigram overlap} > 0.28)$$

  1. The classifier — 16 features → {good, partial, memorized, wrong}.

Training (Python, offline). training/features.py mirrors scoring.ts line by line — same thresholds, same NLI gating — so features extracted in Python transfer exactly to TypeScript inference. We train an ExtraTrees model, export it to weights.json, and re-implement the tree walk in TypeScript. verify_tree_export.mjs ensures both paths produce identical predictions. No Python at runtime, no ONNX, no server. The architectural rule everything else follows from: the engine feeds the LLM, never the reverse. Gaps and incorrect statements flow into the persona prompt. Nothing flows back. Remove the API key and the personas degrade to a simple notice while every score is still computed. The score isn't a chatbot's opinion — and we can prove it by unplugging the chatbot.

Stack: Next.js 16 (App Router) · React 19 · Tailwind v4 · Framer Motion · transformers.js for both models (no inference server) · Zod on every API boundary · provider-swappable LLM module with a model fallback chain, because free-tier quotas are per model and a 429 shouldn't end a demo.

Challenges we ran into

NLI contradiction is meaningless between unrelated sentences. Our first implementation compared every clause against every reference fact and produced confident nonsense: "oxygen is released" vs "glucose stores chemical energy" scored as contradiction $0.99$, because the model was trained on same-scene premise/hypothesis pairs. Fix: gate NLI behind embedding similarity $\geq 0.45$ and compare each claim only to its closest matching fact. A modelling assumption we discovered by reading garbage output.

Over-eager clause splitting broke everything downstream. Splitting on " and " shredded "carbon dioxide and water" into "oxygen." — a fragment whose entailment score was pure noise. The whole pipeline was poisoned by a one-character regex.

The coverage threshold is a real precision/recall trade-off. At $0.45$, Newton's third law counted as "covered" without mentioning action–reaction. At $0.55$ it's correctly flagged as a gap, but genuinely covered nodes can also land around $0.55$. Generic MiniLM similarity simply isn't calibrated for "did this person explain this idea?", making this our top roadmap item.

good vs partial was a data problem, not a model problem. The first model scored 0.733 held-out, with most errors on that boundary. Our generator built both classes from the same template, differing only in missing nodes, so there was no real distinction to learn. Replacing it with LLM-authored explanations across multiple subjects and adding distribution-shape similarity features (min, p25, median, std, weakest_two_mean) raised held-out accuracy to 0.90.

Free-tier LLM quotas are per model, not per key. We discovered this mid-demo. Now a fallback chain switches models on 429 instead of failing.

Accomplishments that we're proud of

Current model performance:
Samples / features / classes 3,219 · 16 · 4
5-fold CV accuracy (ExtraTrees) 0.889 ± 0.017
Held-out accuracy 0.904
memorized F1 0.97
wrong F1 0.98
Misconception-category classifier (5 classes) 0.960 held-out

The ablation is the result we're proudest of, because it validates a design decision rather than just reporting a score. Removing the two copying features gives:

Metric Full Ablated Δ
Held-out accuracy 0.904 0.846 +0.058
memorized recall 0.944 0.790 +0.153

Semantic similarity alone cannot distinguish a memorized answer from a truly understood one, because memorized explanations are still semantically correct. trigram_overlap became the model's highest-importance feature (0.175). The two-signal design isn't just a story — it's an empirical requirement.

Other things we're proud of:

  • The score survives with the LLM switched off. Not a fallback — an argument.
  • Python↔TypeScript feature parity, verified by tests.
  • Warm analysis in ~0.5 s (252 ms embedding + 269 ms NLI), fully local.
  • Confidence calibration that measures the illusion of competence directly.
  • A teacher dashboard that reports what the class missed, never a leaderboard.
  • Complete accessibility without voice. Voice is optional, never required.

What we learned

Data quality beat every modelling decision. The +11 point improvement came from replacing the data generator while keeping the feature set unchanged.

Pre-trained models have hidden assumptions. The NLI cross-encoder returned confident but incorrect scores because we fed it sentence pairs outside its training distribution. Reading raw outputs exposed what tests didn't.

Ablation tells you whether a feature is a story or a signal. The Parrot Detector wasn't just a good demo — it improved recall on the class it was built to detect by +0.15.

Architectural constraints became product features. "The LLM never touches the score" started as engineering discipline and became the reason teachers trust the system.

Thresholds are where ML meets ethics. Choosing $0.55$ over $0.45$ changes whether a student is told they understand something they don't—or missed something they actually knew. There is no neutral threshold, only a chosen error profile.

What's next for Lucid

  • A fine-tuned embedding model. Train MiniLM on explanation pairs so similarity reflects explanation quality instead of generic topical similarity. Export to ONNX and run in transformers.js.
  • A human-labeled test set. Replace synthetic evaluation with 50–80 real student explanations, independently labeled by two raters with Cohen's $\kappa$ reported.
  • Concept maps from PDFs or syllabi. Generate high-quality concept maps directly from teacher materials.
  • Multilingual support. Expand beyond English to reach more learners.
  • Deeper depth scoring. Improve beyond similarity-based evaluation of two "why" questions.
  • Smarter spaced repetition. Replace fixed score-based intervals (7 / 3 / 1 days) with data-driven scheduling.

Built With

Share this project:

Updates

Submission history