The Multimodal Semester Memory Agent

• 

  This is our user interface

Inspiration

Students today learn from many sources such as lectures, whiteboard notes, PDFs, and recorded sessions, but this information is usually scattered across different devices and apps. When exams approach, it becomes difficult to locate the right concept quickly. We built Syncra to solve this problem by transforming scattered study materials into one intelligent, searchable semester memory that keeps everything organized and easy to access.

What it does

Syncra is a multimodal study assistant that brings lecture audio, whiteboard photos, and PDFs into one unified system. It converts raw lecture content into structured notes, predicts upcoming exam topics using AI, and generates visual explanations including 3D learning aids. The platform allows students to search their entire semester’s content like a memory engine, making revision faster and more efficient.

How we built it

We built Syncra using a modern TypeScript-based web frontend connected to backend AI processing modules. The system uses the Gemini model for multimodal understanding and prediction. The core pipeline allows users to upload lecture audio, images, or PDFs, extract text and key concepts, store them in a structured memory, and then use AI to generate summaries, predictions, and visual learning aids.

Challenges we ran into

One of the main challenges was handling multiple data types—audio, images, and PDFs—within a single processing pipeline. We also had to maintain context across an entire semester’s worth of content while keeping the interface simple for students. Another challenge was ensuring fast responses while processing large lecture files and AI-generated outputs.

Accomplishments that we’re proud of

We successfully built a working multimodal semester memory system that integrates AI to predict exam-relevant topics. The platform unifies different types of study materials into a single interface and uses a scalable architecture that can support future features and improvements.

What we learned

Through this project, we learned how multimodal AI can solve real-world education problems and how to structure unorganized learning data into meaningful insights. We also gained practical experience integrating AI APIs into a web application and understood the importance of user experience when designing AI-powered tools.

What’s next for The Multimodal Semester Memory Agent

In the future, we plan to add real-time lecture transcription, a mobile app for on-the-go study, and personalized study plans based on student performance. We also aim to introduce collaboration features for group learning and more advanced 3D and interactive concept visualizations.

Built With

• apis
• css
• gemini-api-(google-ai)
• html
• javascript
• multimodal-ai-pipelines
• node.js
• rest
• typescript