Abhyas

Five Gemini agents that consume any textbook chapter, diagnose why each student got answers wrong (called misconceptions, not red crosses), and create a personalised exam for each student in one click

Comment

Inspiration

A teacher with forty students faces one exam paper but has forty different sets of misunderstandings. Two students can both score 6/10 for completely different reasons; one confuses mass with weight while the other confuses g with G. They both receive the same revision sheet. Every teacher wants to offer personalized help, but few have the time. We built Abhyas (Hindi: practice) so the diagnosis, paper creation, and personalization happen automatically, allowing the teacher's job to shrink to a single click.

What it does

Abhyas is a five-agent system that runs a complete teaching loop:

  1. Ingest — upload any Class 9/10 chapter PDF. It is chunked into concept-tagged units, embedded, and indexed in MongoDB Atlas in about 15 seconds.
  2. Generate — one click creates a proper CBSE-pattern exam: Section A includes MCQs, 2-mark very-short answers, 3-mark short answers, a 5-mark long answer, and a case/activity-based question. Every question is grounded in the uploaded textbook and includes a chapter/page citation that a guardrail checks before the paper is finalized.
  3. Grade — students can solve online (each paper has a shareable link that requires no login and works on a phone) or on paper, which the teacher photographs and uploads. Gemini Vision reads the handwriting and the Grader agent marks each answer, identifying the misconception behind each mistake from a closed taxonomy (mass_weight_confusion, g_vs_G_confusion) rather than just marking it incorrect.
  4. Profile — every graded submission updates the student's profile, including weak topics, recurring misconceptions, and calibrated difficulty.
  5. Personalize — the standout feature: Create Assignment evaluates the whole class and decides for each student automatically. Students with identified weaknesses in that chapter receive a paper tailored to their gaps, while students who have never been tested on it get a general paper. The button responds in under a second, and papers materialize live as agents complete them.

Teachers can also ask questions in plain English, like "which students still confuse mass and weight?" An ADK agent will respond by querying live Atlas data through the MongoDB MCP server, with every agent's step traced in Arize Phoenix.

How we built it

  • Agents: Google ADK (Python) — five specialized sub-agents (ingestion, generator, grader, profiler, personalizer) and a root orchestrator carrying two MCP toolsets (MongoDB + Phoenix).
  • LLMs: Gemini 2.5 on Vertex AI — Flash with tuned thinking budgets for generation and verification, Pro for grading; gemini-embedding-001 at 1024 dimensions.
  • Data & retrieval: MongoDB Atlas as the single store; Atlas Vector Search + Atlas Search (BM25) combined with client-side Reciprocal Rank Fusion for hybrid retrieval; aggregation pipelines power the misconception heatmap.
  • Partner integration: The MongoDB MCP server (pinned, read-only, tool-filtered) provides the agent's connection to Atlas for natural-language questions — it is visible and load-bearing, not just plumbing.
  • Observability: Arize Phoenix Cloud traces every agent's step via OpenInference instrumentation (isolated TracerProvider), and the Phoenix MCP server allows agents to inspect their own traces.
  • App: FastAPI + Next.js, both on Cloud Run; secrets are stored in Secret Manager; Firebase auth for teachers; public no-login solve links for students.
  • Discipline: TDD against the real database and the real model — no mocked MongoDB, no mocked LLM, structural assertions for non-deterministic agent output. Over 100 tests were conducted.

Challenges we ran into

  • The invisible thinking tax. Our papers took over 3 minutes to generate. Profiling revealed that Gemini 2.5 models took between 5 and 15 seconds thinking before every answer by default. We managed this by budgeting thinking time based on the call type, batching generation per exam section rather than per question, over-generating spare candidates to prevent verification-related delays, caching retrieval, and gating all calls behind a global semaphore. This reduced the time for one paper from 95.8 seconds to 16.9 seconds — and an async, progressively revealing UI made the click feel instant.
  • $rankFusion is gated. Server-side hybrid search requires MongoDB 8.1+/M10+. We run Atlas Flex (8.0). We implemented Reciprocal Rank Fusion client-side using the same formula with K=60.
  • Atlas TLS "alert 80". Handshake failures that appeared to be certificate problems were Atlas rejecting non-allowlisted IPs. The pymongo and Node's driver resolved routes differently, meaning one connected while the other did not.
  • ADK v2 landed mid-hackathon with breaking changes just days before we wired the orchestrator. We used v1 and pushed forward.
  • Grounding vs. synthesis. Our guardrail validated each question against its single cited chunk, resulting in the rejection of perfectly good 5-mark and case questions that genuinely synthesize across materials. Checking against the section's full retrieved context resolved this without allowing errors.
  • LLM drift vs. aggregatable data. Using free-form misconception labels would have made the class heatmap impossible. A closed taxonomy, along with a post-LLM guardrail (off-list labels become null), keeps diagnostics both clear and queryable.

Accomplishments that we're proud of

  • A teacher uploads a chapter and, in 60 seconds, has a citation-backed CBSE exam. After one round of submissions, every student's next paper is different, automatically determined.
  • The grader correctly identifies designed misconceptions: a student answering "things weigh less on the Moon because there's no air" receives the label gravity_needs_atmosphere rather than just losing 5 marks.
  • Handwritten answer sheets photographed on a phone are graded with identified misconceptions in about a minute.
  • The grounding guardrail prevented an unanswerable question from being sent during our demo, demonstrating quality enforcement in action.
  • A 5.7× reduction in latency with no loss of structure, citations, or verification.

What we learned

Agentic depth is more about discipline than a collection of features: closed taxonomies work better than clever prompts, structural tests against real services are superior to using mocks, latency can often hide in invisible defaults (thinking budgets!), and the most valuable MCP integration is one that a judge can observe answering a real question against a real database.

What's next for Abhyas

We aim for multi-subject and multi-grade coverage on a larger scale, batch OCR for whole-class photo uploads, a UI for teachers to edit questions, an open misconception taxonomy that evolves with new error patterns, moving the orchestrator to Vertex AI Agent Engine as a dedicated managed runtime, and launching a real classroom pilot.

Built With

Share this project:

Updates