Lumen

A note-taking web app that helps students see what they're learning. Highlight any concept in your notes, hit Animate, and a 3Blue1Brown-style explainer plays inline, generated on the fly.

Inspiration

Captions were built for deaf users. Now everyone uses them. We kept coming back to that pattern: accessibility tools become universal tools.

We've all reread a paragraph three times and still not gotten it. Pre-med students grinding through biochem, career-switchers picking up calculus, international students parsing dense English textbooks, the bottleneck isn't effort, it's that text alone is a low-bandwidth way to transmit hard ideas. A 30-second animation of merge sort splitting and merging arrays does what a paragraph can't.

Lumen started as an accessibility tool for students who learn better visually. Like captions, it turned out to be useful for everyone.

What it does

Lumen is a Notion-style note-taking app with one headline feature: highlight a concept, click Animate, and an animated explainer renders inline in your notes.

Highlight → Animate. Select "merge sort," "chain rule," "torque," or any concept. We generate Manim code for it on the backend, render it, and embed the video in the document where you highlighted.
Rich-text editor. Notion-style block editor where notes, math, images, and animations live in one place.
PDF / image import. Snap a picture of a textbook problem or upload a PDF, Gemini Vision OCRs the content and drops it into your notes as editable text.
Audio → notes. Record a lecture and get a structured, summarized note back automatically.
Research sidebar. A small panel that surfaces the cognitive science behind visual learning, dual coding theory, and why animations stick when paragraphs don't, so users understand why the tool works, not just that it does.

How we built it
Frontend: React + Vite, with Tiptap as the rich-text editor (block-based, extensible, plays well with custom embeds for the animation player).
Notes backend: Express.js + SQLite. Lightweight, fast to iterate, no infra overhead during the hackathon.
Animation service: A separate Python Flask service that takes a concept string, prompts an LLM to write Manim code, executes it in a sandboxed render, and returns the resulting MP4.
AI layer: OpenRouter for the code-generation step (Manim scene generation), Gemini Vision for OCR on PDFs and image uploads, and a transcription + summarization pipeline for the audio-to-notes feature.
Glue: The frontend hits the Express API for notes and a separate endpoint for animation jobs. Renders are async, the editor shows a placeholder while Manim is working, then swaps in the video once it's ready.

Challenges we ran into
LLM-generated Manim code doesn't always compile. Manim has a specific API surface, and LLMs hallucinate methods that don't exist or use deprecated syntax from old Manim Community versions. We added a retry loop: if the render fails, we feed the traceback back to the model and ask it to fix the code. Most failures resolve in one or two retries.
Render times. Even simple Manim scenes take 10–30 seconds to render, and complex ones blow past that. We had to make the UX async-first, a placeholder appears immediately, the user keeps writing, and the animation fills in when it's ready. We also cache renders by concept so the second person to animate "merge sort" gets it instantly.
Concept extraction from a highlight. Users highlight messy fragments, full sentences, partial phrases, sometimes a single word with no context. We had to prompt the model to first interpret what concept the user actually wants animated before generating any code.
Sandboxing Manim execution. Running LLM-generated Python on a server is a security minefield. We isolated the renderer in its own service with strict resource limits and no filesystem or network access beyond what Manim needs.
Editor ↔ embedded media. Getting Tiptap to treat a rendered animation as a first-class block (movable, deletable, undo-able) without breaking the rest of the editor took longer than expected.

Accomplishments that we're proud of
The highlight-to-animation loop actually works end-to-end. Highlight a concept, wait, watch it animate inline. The first time we saw "chain rule" render correctly was the moment the project felt real.
The retry-on-compile-error loop turned a flaky feature into a reliable one.
Caching dropped repeat-render latency from ~20s to instant, which makes the tool feel genuinely usable in a study session.
We shipped four input modalities, typing, pasted text, PDF/image OCR, and audio transcription, into one coherent editor.

What we learned
Latency shapes UX more than features do. A 20-second blocking animation render kills the tool. The same render, async with a placeholder, feels fine. Same code, different UX, completely different product.
LLMs are bad at writing code in niche libraries. Manim is well-documented but small. Generic code generation prompts fail constantly. Tight prompts with examples, version pinning, and a retry-on-error loop turned a 30% success rate into something usable.
Accessibility framing sharpens the design. Building "for students who struggle with text" forced clearer choices than building "a cool note-taking app" would have.
Sandboxing is non-optional. Executing model-generated code on your server without isolation is how you end up on a postmortem blog post.

What's next for Lumen
Faster renders. Pre-rendering common concepts (top 500 high-school and intro-college topics) so the first animation a user ever generates is instant.
Interactive animations. Move from MP4 output to embedded Manim/web-rendered scenes the user can scrub, pause, and parameterize (change the input array, change the function being differentiated).
Spaced repetition. Notes plus animations plus a review schedule, turn passive notes into active recall.
Collaboration. Shared notebooks for study groups, with a library of class-specific animations that grows as the group uses it.
Mobile capture. A phone app focused on the camera-to-notes flow: snap a textbook page, get parsed notes with animatable concepts already highlighted.