OmniTutor AI

Inspiration

Inspiration

Traditional voice assistants answer questions with static summaries, but they fail to act dynamically when learning complex engineering and technical concepts. With the evolution of Alexa+, voice agents need to transition into proactive reasoning partners capable of calling external tools, tracking milestones, and personalizing explanations. We built OmniTutor AI to demonstrate how an agentic Alexa+ interface can turn voice interactions into interactive tutoring, instant code diagnostics, and structured study planning.

What it does

OmniTutor AI simulates a next-generation Alexa+ interface using the Model Context Protocol (MCP) paradigm and AWS services:

• Interactive Conversational Study: Users talk or type naturally. OmniTutor breaks down difficult technical subjects dynamically.
• Autonomous Tool Execution (Simulated MCP):
  • Quiz & Diagnostic Generator: Dynamically creates targeted review questions to check comprehension.
  • Code & Logic Analyzer: Parses user-supplied code snippets, detects logic flaws, and provides guided, step-by-step corrections.
  • Milestone & Schedule Planner: Automatically decomposes assignment briefs and project goals into actionable daily task checklists.
• Voice In / Voice Out Experience: Features speech-to-text input and natural text-to-speech audio feedback, mirroring the Alexa+ hardware experience directly in a responsive web app.

How we built it

• Frontend: React with Tailwind CSS, featuring real-time audio waveform visualizers, voice-toggled speech synthesis (Web Speech API), and reactive tool execution cards.
• Backend Architecture: Python FastAPI server orchestrating agentic flows.
• Tool Protocol: Implemented structured tool definitions adhering to the MCP pattern, letting the agent dynamically choose between analysis, quiz generation, and scheduling tools.
• Cloud & AI Layer (AWS Builder): Integrated with Amazon Bedrock via the AWS SDK (boto3) to power high-speed reasoning, intent categorization, and tool call payload generation.

Challenges we ran into

• Strict Latency Demands: Ensuring round-trip voice recognition, tool selection, reasoning, and speech synthesis responded in under two seconds.
• Deterministic Tool Calling: Fine-tuning prompt schemas so the agent accurately outputs valid JSON payloads for external tool execution without hallucinations.
• Simulating Alexa+ Constraints: Accurately replicating multi-turn conversational memory and device state entirely inside a standalone web application.

Accomplishments that we're proud of

• Delivered an end-to-end working voice and text simulator that showcases true agentic behavior rather than simple prompt/reply chat.
• Successfully connected structured tool execution with Amazon Bedrock within a tight hackathon timeline.
• Designed an intuitive UI that gives clear visual confirmation whenever the assistant triggers an autonomous tool behind the scenes.

What we learned

• Implementing the Model Context Protocol (MCP) design pattern drastically improves agent reliability compared to standard prompt chaining.
• Designing for voice requires handling interruptions, ambiguous spoken syntax, and keeping audio summaries concise.

What's next for OmniTutor AI

• Adding native Alexa Skill Kit (ASK) deployment alongside the MCP simulator once developer preview access opens.
• Integrating real-time file uploads (PDF lecture notes and textbook chapters) using Amazon Bedrock Knowledge Bases.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for OmniTutor AI

Built With

• amazon-web-services
• amazonbedrock
• css
• fastapi
• mcp
• python
• react
• tailwind
• webspeechapi