Inspiration

You shouldn’t have to rewrite your course because a browser agent can finish your quiz.

Professors already write assessments, teach the material, answer questions, and grade. Now a student can hand an entire quiz to an AI agent and submit a perfect score without doing the reasoning the assignment was meant to teach. When that happens, the student misses the practice, and the professor loses a chance to see what the class is struggling with. Asking every professor to redesign every quiz or learn a new strategy for spotting AI adds another job.

We wanted to give professors their time back. Start with the quiz you already have in Canvas. Unprompted searches for questions online, benchmarks what agents can solve, drafts replacements that stay within the skills the original quiz tests, and handles the first follow-up with the student. The professor keeps control of what gets published and can see exactly what each proposed change preserves. The workflow fits around the existing course, syllabus, and Canvas export.

What it does

For the professor. Import a Canvas QTI quiz, paste a quiz in Markdown, or scan a public course page you own. The studio shows which questions already appear on homework-help sites, with links and evidence, and which ones AI agents solve in a sandbox. Review candidate rewrites beside their answer, skill, and search checks. Approve the versions you want students to see and export the result as a Canvas-importable QTI ZIP. Published quizzes can be rechecked for new leaks over time.

How Neon, Gemini, Bright Data, and Mastra verify quiz rewrites

For the student. Take the quiz in a chat. On one or two selected questions in a typical attempt, the student rates their confidence, explains their reasoning, answers a personalized what-if, and rates their confidence again. The system gives more weight to questions with higher agent solve rates or known leaks and includes some randomness in the selection. There’s a typed path and an accommodation flow for questions that depend on a figure. The progress page turns the session into a roadmap of quizzes and skills to practice.

A student takes a practice quiz and starts the think-aloud

What that sounds like. In our statistics quiz, a student correctly answers 1/6 for the probability of rolling a total of seven on two rolls of a fair die. After hearing how they counted the six matching pairs, the coach asks what changes if the target is eight. The student can explain that five of the 36 ordered pairs now work. The professor gets to see the reasoning behind the original score.

ElevenLabs Explain Coach asks a personalized what-if question

For review. The instructor sees answer correctness alongside explanation evidence, confidence changes, and session signals. A separate grading council gives a proposed 0–4 explanation score with quotes from the student’s own words. Three Gemini models score independently, reconsider once after seeing each other’s reasoning, and return the median. The rubric evaluates reasoning, and the professor controls approval. Session signals suggest where to follow up. Students can review feedback and practice the skills that need more work.

Professor Insights shows the reasoning behind the score

Bright Data for Good · make exposed questions actionable

We turned the open web into evidence a professor can use. Unprompted uses Bright Data's SERP API to look for distinctive wording from each quiz item on public homework-help sites. A professor sees the matching source, a snippet and its match strength, and can inspect a screenshot when the page is publicly available. A second Bright Data search tests candidate rewrites for existing matches. Archive-backed leak watch helps surface questions that appear online after a quiz is published.

We used the same pipeline to build the Textbook Leak Index, a public study of 500 randomly sampled exercises from ten OpenStax textbooks. We ran 697 searches and separately tracked unresolved cases. Results showed the actual question for 180 exercises (36%), and 242 (48%) met that criterion or matched an exact phrase. The open dataset publishes source links and match levels. Textbook wording was used for search, and our Gemini benchmark used quizzes we wrote. Anyone can check their own question or download the data and inspect the method.

Bright Data Textbook Leak Index · 500 OpenStax exercises

For a professor, this turns “my quiz might be online” into a reviewable list of items, with evidence and a concrete path to a fresh version. The same evidence helps prioritize where a short student explanation would add the most value.

Meet the agents

Eight roles work across the student app and professor studio. Mastra powers three helper agents and the four named roles in the rewrite workflow. ElevenLabs runs the Explain Coach. The browser agents that benchmark our quizzes run separately.

Eight roles across Mastra agents, the rewrite workflow, and ElevenLabs

Agent or role What it does
Tutor Presents questions one at a time, checks submissions against the server’s grading tools, and explains answers after submission in practice mode. It can also serve practice questions grounded in dated, cited data.
Researcher Answers student “why” questions with cited public sources found through Bright Data.
Leak Scout Summarizes existing matches for a professor and can run a fresh search for quiz questions. It reports the source and the evidence for the match.
Explain Coach Runs the ElevenLabs conversation. It asks the student to explain their answer, calls the app for a Gemini follow-up grounded in what the student just said, and records the exchange for review with consent.
Quiz Checker Brings together leak evidence and server-graded browser runs. The rewrite workflow can prioritize questions found online or solved by agents at least 60% of the time.
Rewriter Proposes new versions of an existing question by changing its numbers, scenario, or presentation while keeping the original learning objective in view.
Answer Checker Solves each candidate three times with a code interpreter and checks that the result agrees with the proposed answer key every time.
Judge Panel Uses three Gemini models to check whether the candidate tests the same skill and has unambiguous wording. This panel reviews questions. The grading council reviews student explanations.
Browser agents Use browser-use or Stagehand to take controlled quizzes. The server grades their submitted answers, and the studio reports solve rates, confidence intervals, and excluded runs.

Agentic memory management

The app passes Mastra up to 20 recent messages along with the quiz and attempt context for each turn. The server looks up the attempt to determine whether it is practice or graded, and that mode controls which tools are available. Mastra also saves rewrite workflow state in Neon, so an interrupted run can resume with the same candidates and checks when the professor makes a decision.

How we built it

Next.js 16 and TypeScript serve the student chat, professor studio, and public tools. Neon Auth handles sign-in for the student and professor experiences. The benchmark has its own quiz sandbox, which lets us run agents against quizzes we control and grade submissions independently of what the agent reports.

Unprompted architecture

Technology Where it earns its place
Bright Data SERP API for question matches, Browser API for screenshots of public evidence, and Web Archive for leak-watch comparisons. The Researcher also uses Bright Data to find public sources students can check.
Gemini + Mastra Gemini powers the browser benchmark, rewrite generation, question judges, and the what-if prompts. Mastra coordinates the transform stages, persists workflow state, and pauses verified candidates for instructor approval. The agent panel streams progress and check results into the studio.
Neon Stores quizzes, attempts, and review records. Each rewrite experiment gets its own copy-on-write branch, so the professor can inspect its history while the delivered quiz stays stable. The appeal flow uses time travel to show the quiz version a student received.
ElevenLabs Handles the live voice session and the tool call for the personalized follow-up. Scribe word timestamps help measure how long a student took to start answering that follow-up. We exercised the live agent’s tool loop in text mode.
Tiger Data Stores time-series interaction events and rolls up signals such as clicks, typing, and focus changes. In our benchmark, interaction traces surfaced patterns missed by conventional browser bot checks. Professors get evidence to review alongside the explanation.
Solana devnet Implements a public integrity check for file fingerprints. The design keeps student data off-chain and protects answer-key fingerprints with server-keyed hashing. The public verification endpoint currently shows the Leak Index CSV as pending anchoring.
Vultr + custom domain Vultr hosts the Next.js app and Mastra workflow service behind Caddy. The live site, public checker, and study are at unprompted.courses.

A rewrite gets a branch, a proposed answer, independent solves, and objective and wording checks. A known online match rejects the candidate. A search that fails to run is recorded as unchecked. When an agent retake produces valid submissions, its solve rate appears beside the other checks. The default workflow can surface a verified candidate for professor review even if an agent can still solve it. Publishing a replacement requires professor approval.

What we measured at SanD Hacks

  • 175/175 correct answers across six original quizzes. In valid sandbox submissions, browser-use agents powered by Gemini got statistics 25/25, statistical inference 36/36, calculus 36/36, Python 30/30, biology 24/24, and microeconomics 24/24. We separately recorded 14 infrastructure failures.
  • Ten rewrites of one question still left us with the real problem. We tried changing numbers, moving the question into a new setting, asking students to find an error, and making them read a table. Agents solved all four versions with recorded valid retakes, with results of 6/6, 6/6, 8/8, and 8/8. That experiment made the explanation step central to the product.

The hard part was preserving what a professor meant to test while changing how a question was asked. We built the import/export loop, the isolated benchmark, the branch-per-candidate verification workflow, the voice and typed runtime, the professor review tools, and the public study in the same hackathon. The result gives a professor a practical way to start with Monday’s quiz and see where students need help.

Professor Studio replays a Gemini browser-agent benchmark

What’s next

Pilot Unprompted with instructors and students. Measure how much preparation time the workflow saves, how students experience the think-aloud, and whether explanation evidence helps instructors give better feedback.

Try it

Built With

Share this project:

Updates

Submission history