Astray: Visual AI Tutor

Most tutors just give you the right answer. Astray watches how you actually solved it, finds your specific mistake, and builds a short video that shows you exactly where your reasoning went wrong.

Comment

Inspiration

Getting a wrong answer marked wrong doesn't tell you anything. You still don't know where your thinking actually broke. We wanted a tutor that finds that exact spot and shows it, instead of just handing back a corrected answer.

What it does

Submit your work, typed, photographed, or spoken out loud. Astray finds the specific misconception behind the mistake, checks that diagnosis with real symbolic math instead of trusting a model's opinion of itself, and builds a short animation around that one error. You can ask it questions afterward, and the answers point to exact moments in the video. Click a citation, the player jumps there. It also remembers which mistakes keep showing up across different students.

How we built it

FastAPI and SQLite on the backend. DeepSeek handles diagnosis, checked against a seeded taxonomy and verified with SymPy so a confident-sounding answer still has to be right, not just plausible. Photo submissions go through Gemini for transcription, normalized to consistent LaTeX so formatting never depends on the model guessing right twice in a row. The animation side runs a staged pipeline shaped like Math-To-Manim's approach: intent, prerequisites, curriculum, storyboard, scene code, validation, render. It starts from a diagnosis instead of a raw question, though, and the generated code runs sandboxed behind an AST allow-list inside Docker instead of being trusted outright. Every animation beat gets a measured timestamp, which is what makes the chat citations actually land on the right second instead of approximating. Narration comes from fish.audio, timed to each beat's real duration. Voice input goes through the browser's speech recognizer with a wake word and a router that tells a new problem apart from a follow-up. No frontend build step. Plain JS and CSS.

Challenges we ran into

The diagnosis logic worked early. What broke were the parts that only show up once the system runs for real. Correct work was quietly minting fake misconceptions and polluting our own pattern data. A missing ORDER BY meant newly created misconceptions could never be matched again. A verification check could report full confidence on something that was true by definition and could never fail. In the animations, a leftover title card once rendered directly over the one frame the whole video existed to show, and a Manim alignment bug made two lines of text silently overlap because the alignment axis matched the direction they were stacked in. None of this showed up in a test suite. It only showed up from running the actual pipeline and looking at what came out.

Accomplishments that we're proud of

Ask the tutor a question, click the citation it gives you, and the video seeks to the correct second. That's measured, not eyeballed. And when a student genuinely doesn't know where to start, the system teaches the method instead of inventing a mistake to correct. We found it doing exactly that once, fabricating an error that never happened, and fixed it before it could mislead anyone.

What we learned

A green test suite tells you the code runs. It doesn't tell you the product is honest. Every bug that actually mattered here (the fake data, the unreadable frame, the false confidence score) only turned up when we ran the real thing and checked it the way a skeptical user would, not the way a developer checks a diff.

What's next for Astray

A larger misconception library, a teacher-facing view of the pattern data, and expanding voice beyond push-to-talk into always-on listening.

Built With

Share this project:

Updates

Submission history