• 

  Data Flow Diagram

The Problem

Students often receive a score after completing an assessment, but very little guidance on why they lost marks or how to improve. Static feedback is difficult to engage with, and reading long evaluation reports can feel overwhelming. Many learners would rather have a natural conversation where they can ask follow-up questions, clarify concepts, and receive personalized explanations.

Who It's For

This voice learning companion is built as an extension of Assessly for:

• Students preparing for exams and competitive tests
• Universities conducting rubric-based assessments
• Teachers who want scalable, personalized feedback for every student
• Self-learners looking to understand mistakes instead of just viewing scores
• Educational institutions adopting AI-assisted learning workflows

What We Built

We built an AI-powered voice learning companion that transforms assessment results into an interactive tutoring experience.

After a student completes an assessment in Assessly, the platform automatically generates detailed learning feedback based on their submitted answers, evaluation rubric, and scoring results. Instead of reading a static report, students can start a natural voice conversation with an AI tutor that explains mistakes, answers follow-up questions, revisits difficult concepts, and provides personalized guidance for improvement.

Rather than replacing teachers, the system acts as an always-available learning assistant that turns assessment into an opportunity for continuous learning.

How We Built It

Multi-Agent AI Architecture

We designed the system using six specialized AI agents, each responsible for a specific stage of the learning pipeline.

• Assessment Analyzer extracts performance patterns from graded submissions.
• Feedback Generator produces structured, rubric-aware explanations.
• Concept Tutor explains incorrect answers and underlying concepts.
• Question Generator creates personalized follow-up practice questions.
• Progress Evaluator identifies strengths, weaknesses, and learning gaps.
• Voice Tutor uses Gemini Live to deliver all feedback through natural, conversational voice interactions.

Backend Infrastructure

Built using FastAPI, the backend orchestrates communication between all AI agents while managing assessment workflows. Firestore stores assessment data, generated feedback, conversation history, and learning progress. WebSockets enable low-latency, real-time communication for voice interactions.

Frontend Development

The frontend was built with Flutter Web, providing a responsive interface where students can review assessment results, interact with the AI tutor through voice or text, and continue personalized learning sessions.

Voice Processing

Voice conversations are powered using the Gemini Live API. Audio is captured through the MediaRecorder API, streamed using WebSockets, and processed in real time, allowing students to interrupt, ask follow-up questions, and receive immediate spoken explanations in a natural conversational flow.

Learning Workflow

Once an assessment is graded, the system automatically analyzes rubric-level performance, generates personalized learning feedback, identifies weak concepts, prepares follow-up explanations, and makes everything accessible through an interactive voice tutor instead of a static report.

🤖 How We Used Claude

Claude powered the reasoning capabilities across multiple specialized agents.

Agent	Model	Purpose
Assessment Analyzer	Claude Haiku	Efficient analysis of graded responses
Feedback Generator	Claude Sonnet	Personalized learning feedback
Concept Tutor	Claude Sonnet	Explains mistakes and difficult concepts
Practice Generator	Claude Sonnet	Creates targeted follow-up questions
Progress Evaluator	Claude Sonnet	Identifies strengths, weaknesses, and learning gaps

Gemini Live handled real-time conversational voice interactions, enabling students to naturally discuss their assessment results with the AI tutor.

Accomplishments We're Proud Of

• Transforming static assessment reports into interactive learning conversations.
• Building a multi-agent architecture where specialized AI systems collaborate to generate personalized educational feedback.
• Delivering low-latency voice interactions that feel like talking to a real tutor instead of navigating a chatbot.
• Creating a scalable workflow that helps students understand mistakes rather than simply displaying scores.

What We Learned

• Designing AI systems where multiple specialized agents collaborate produces significantly richer educational feedback than relying on a single large prompt.
• Real-time voice systems require careful optimization of streaming, latency, and interruption handling to create natural conversations.
• Educational AI is most valuable when it encourages understanding and improvement rather than simply automating grading.
• Modular testing of voice streaming, agent orchestration, and feedback generation made the system significantly more reliable during development.

What's Next

Our next goal is to evolve the voice tutor into a complete AI learning companion.

Future improvements include adaptive learning paths based on assessment history, long-term progress tracking, personalized revision plans, multilingual tutoring, and integration with course materials so students can ask questions directly about lectures, notes, and textbooks. We also plan to make conversations more context-aware, allowing the tutor to remember previous learning sessions and guide students through continuous improvement across multiple assessments.

Built With

• claudeapi
• dart
• deepgram
• elevenlabs
• flutter
• tavily
• websocket