Inspiration

Have you ever sat in a university lecture, completely lost, but felt too embarrassed to raise your hand and ask a question?

We realised that this is a common but often invisible problem in higher education. In large lectures, students may struggle to understand complex concepts, unfamiliar terminology, or everyday English expressions, especially international students. Yet lecturers rarely know when or why students are confused. By the time module surveys or exam results reveal the problem, it is often too late to address it.

We wanted to change this. Our idea was simple: What if no student's confusion ever went unheard?

That question inspired us to build Echo, an AI-powered lecture companion that helps lecturers anticipate confusion, understand students' difficulties in real time, and improve their teaching after every class.

What We Built

Echo is a web application designed around the entire lecture journey: Before class, During class, and After class.

1. Before Class: Predict with ClassTwin

Before a lecture, lecturers upload their presentation slides as a PDF. Five AI student agents, each representing a different learning background, analyse the slides in parallel.

For example, our virtual students include an international student, a student with weaker mathematical foundations, someone without prerequisite knowledge, a student who learns better through examples, and a high-performing student.

Each agent identifies potentially confusing slides and assigns a confusion score from 0 to 5. An advisor agent then combines their feedback into practical teaching suggestions, such as clarifying unfamiliar terminology, adding missing explanations, or introducing intuitive examples.

Students also receive a preparation pack containing a glossary, recommended revision topics, and the slides most likely to be challenging.

2. During Class: Listen with Echo Live

During lectures, Echo provides live captions and explains difficult terminology and British English expressions in simple language. Students can also access explanations in their own language.

More importantly, students can anonymously tap an "I'm Lost" button or submit questions without having to speak in front of a large audience.

Our AI system analyses these interactions and groups similar questions into common confusion topics. Instead of receiving a long list of individual messages, lecturers see the main issues students are struggling with, together with suggested explanations they can use immediately.

3. After Class: Learn and Improve

After each lecture, Echo compares its pre-class predictions with actual student questions and confusion signals.

This creates a feedback loop: missed predictions help improve the virtual student profiles for future lectures, while students receive a personalised recap based on the questions they actually asked.

Echo doesn't just identify confusion; it continuously learns where students need more support.

How We Built It

We built Echo as a web application using Python, Streamlit, Lovable, and Claude.

At its core is a multi-agent architecture in which each AI agent has a specific responsibility:

  • Five student agents: Analyse lecture slides in parallel from different student perspectives.
  • Advisor agent: Consolidates the predictions and generates actionable teaching recommendations and student preparation materials.
  • Speech-to-text and term explanation components: Process lecture audio, generate captions, and identify terminology that needs clarification.
  • Clustering agent: Groups anonymous student questions and confusion signals into meaningful topics for lecturers.
  • Summary agent: Generates post-lecture recaps and helps update the virtual student profiles based on prediction errors.

We focused on making the different components work together as one continuous learning cycle rather than building another standalone transcription or polling tool.

What We Learned and What We Achieved

One of our biggest insights was that identifying how many students are confused is not enough. Understanding why they are confused is what enables meaningful improvements in teaching.

To evaluate Echo, we tested it on a real four-minute university statistics lecture recording.

Our system identified:

  • 22 terms that needed explanations, with definitions in both simple English and Chinese.
  • 8 terms that the lecturer mentioned but never explicitly defined.
  • 7 everyday-word traps, such as "mean", "mode", and "fair", whose ordinary meanings could mislead students in an academic context.
  • 12 unclear moments in the audio, including cut-off sentences.

We also evaluated ClassTwin's predictions against the actual confusion signals from the lecture. It correctly predicted 3 out of 4 major confusion hotspots, including the explanation of "ballpark figure" and why statistical variance is divided by (n-1).

These results demonstrated how AI could connect pre-class preparation with real classroom feedback.

Challenges We Faced

One of our main challenges was making AI understand learning difficulties from different student perspectives. A slide that seems straightforward to a lecturer may be confusing to someone unfamiliar with the terminology or prerequisite knowledge. We addressed this by creating five distinct virtual student agents rather than relying on a single general-purpose AI analysis.

Another challenge was distinguishing genuine academic confusion from ordinary language misunderstandings. Words such as "mean" and "mode" have everyday meanings but specific statistical definitions. We therefore designed Echo to identify these potential misunderstandings and provide context-aware explanations.

We also needed to turn unstructured classroom feedback into something genuinely useful for lecturers. Simply counting anonymous confusion signals would not explain what needed to be taught again. Our clustering approach groups related questions and connects them to specific moments in the lecture, allowing Echo to suggest concrete teaching interventions.

Finally, building a complete before-during-after workflow within a Hackathon timeframe was challenging. We prioritised an integrated prototype that demonstrates how the different AI agents work together, while leaving deeper platform integration and expanded lecture analysis for future development.

What's Next?

We envision Echo being piloted in a large first-year university module over an entire semester.

Our next steps include integrating Echo with university learning platforms such as Blackboard and Canvas, automatically analysing newly uploaded lecture slides, and extending our analysis to visual content such as charts and diagrams.

Ultimately, we want to make university learning more inclusive, responsive, and accessible.

No student's confusion should go unheard.

Built With

Share this project:

Updates

Submission history