StudyPilot - Adaptive Voice AI Agent via MCP

An autonomous, voice-ready Adaptive AI Agent built on MCP that senses changing constraints, dynamically rebalances study schedules, and verifies outcomes without hallucinations.

Comment

Inspiration

Most AI agents and study planners fail because they are completely static. When a student plans to study 2 hours every day, life inevitably happens—they get sick on Day 2, urgent commitments arise, or a complex concept takes twice as long to master. Traditional calendars break, creating an overwhelming guilt spiral that causes students to abandon their goals. Generic LLM chatbots make this worse by hallucinating math and dates without maintaining real state.

We asked: What if a voice AI agent could act as a true adaptive companion? Instead of breaking when life happens, it dynamically senses changes in student availability, rebalances remaining workloads via structured tools, and guarantees mathematical truth before speaking. This inspired us to build our Adaptive AI Agent architecture using the Model Context Protocol (MCP), demonstrated through StudyPilot.

What it does

StudyPilot is an autonomous, voice-enabled learning platform powered by our Adaptive Agent:

  • Dynamic Schedule Rebalancing: When a student's availability changes ("Change the remaining plan to 1 hour per day"), the agent invokes adapt_study_plan to recalculate remaining hours across future days and issues real-time Capacity Warnings if remaining work exceeds available time.
  • Ambient Voice Interaction: Equipped with live Web Speech Recognition and dual MCP transports (STDIO and Streamable HTTP), students can log sessions hands-free using natural shorthand ("day 1 completed 2hrs").
  • Zero-Hallucination Progress Tracking: All plan arithmetic, inclusive date spans, and session logs are computed deterministically via 13 Model Context Protocol (MCP) tools.
  • Active Concept Mastery & Quizzing: Generates tailored multi-choice quizzes (create_quiz), tracks scores (save_quiz_result), and diagnoses weak topics scoring under 70% (get_weak_topics).
  • Dual-Persona Tutoring: Provides pedagogical tutoring on complex computer science, math, and engineering topics (e.g., Normalization, Deadlocks, Recursion) without confusing conceptual questions with schedule mutations.

How we built it

  • Model Context Protocol (MCP): Built with Node.js (ES Modules) using @modelcontextprotocol/sdk. We implemented dual-transport support: STDIO for local agent orchestration and Streamable HTTP (/mcp) with Bearer token authentication for remote Alexa+ assistants.
  • 3-Tier Grounding Engine:
    • Tier 1 (Argument Normalizer): Normalizes parameters and prevents metric terms (like "remaining" or "hours") from being misparsed as IDs.
    • Tier 2 (MCP Domain Invariants): Enforces strict calendar math, leap years, and cognitive daily limits (≤12h/day).
    • Tier 3 (Consistency Checker): Cross-checks model text against database facts before speaking, replacing generic model chatter with verified metrics.
  • Model-Agnostic Core: Designed an abstract BaseProvider contract supporting Groq Cloud (qwen/qwen3.8-27b, openai/gpt-oss-20b), local Ollama (llama3.2), Amazon Bedrock, and an offline fallback provider.
  • Glassmorphic Frontend: Built with React and Vite, featuring dark-mode glassmorphism, activity execution traces, and real-time voice microphone input.

Challenges we ran into

  • Voice Hallucination Benchmark: Voice assistants cannot afford to speak fake dates (like "Feb 30") or bad math. We solved this by strictly decoupling LLM reasoning from state mutation—the AI model is never allowed to calculate hours or edit state directly.
  • Token Limits in Multi-Turn Voice Sessions: Multi-turn dialogue with 13 tool schemas quickly bloats token usage. We implemented sliding-window history pruning to cap conversation context at the most recent 8 messages, keeping requests fast (~1.5s) and well within token budgets.
  • Ambiguity in Natural Voice Shorthand: Users speak loosely (e.g., "add day 2 1 hr"). We built resilient regex normalizers to align natural voice phrasing with strict MCP tool schemas.

Accomplishments that we're proud of

  • 100% Automated Test Pass Rate: 15 out of 15 comprehensive test suites pass cleanly (npm test), verifying contract stability, date math, schema validation, and MCP protocol compliance.
  • True MCP Separation of Concerns: Built 13 full-featured MCP tools that execute over both standard process pipes and remote HTTP microservice endpoints.
  • Zero Silent Failures: When a requested plan exceeds human capacity, the agent provides honest, transparent warnings rather than silently agreeing to an impossible timetable.

What we learned

  • The Model Context Protocol (MCP) is the gold standard for bridging voice AI with external domain logic. Decoupling tool schemas from proprietary LLM prompts prevents vendor lock-in and makes switching models effortless.
  • Anti-hallucination cannot rely on system prompts alone. Deterministic programmatic verification (Tier 3 grounding) is essential for mission-critical voice applications.

What's next for StudyPilot - Adaptive Voice AI Agent via MCP

  • Direct Alexa Skill Deployment: Packaging the Streamable HTTP MCP server into an AWS Lambda function for direct deployment to the Alexa Skill Store and Alexa+ preview devices.
  • Spaced Repetition (SM-2 Algorithm): Integrating algorithmic review intervals into the MCP planner based on quiz performance.
  • Multi-Modal Study Visualizations: Rendering interactive Gantt charts and progress graphs on Echo Show smart displays.

Built With

Share this project:

Updates

Submission history