LabTwin

INSPIRATION

Students learn from textbooks, lecture videos, slides, and programming exercises, but these materials are often scattered across different platforms. General AI chatbots may answer questions without showing where the information came from or understanding a student's weaknesses.

I wanted to build a platform that helps students understand concepts, practice independently, and improve using evidence from their own learning activities.

WHAT IT DOES

LabTwin is a source-grounded AI tutoring and adaptive learning platform built for Track D of the Multimodal AI Hackathon 2026.

LabTwin provides a multimodal knowledge base where teachers can upload PDF notes, presentation slides, and supported lecture videos.

Students can use Ask LabTwin to ask questions and open citations linking to original PDF pages, slides, or lecture timestamps.

The adaptive assessment system supports multiple-choice, short-answer, and numerical questions with source references and feedback.

Personalized learning features include topic mastery estimates, diagnostic assessments, learning recommendations, and a visual mastery map.

The programming laboratory supports Python, C, and Java exercises with hidden test cases, progressive hints, and corrected-code retesting.

Teachers can manage classrooms, create assignments, review student progress, identify misconceptions, and generate reports.

HOW I BUILT IT

The frontend uses React, Vite, JavaScript, HTML, and CSS. The backend uses Python and Django.

The knowledge pipeline extracts information from documents, slides, and supported videos, indexes source content using ChromaDB, and retrieves relevant evidence for tutoring.

LabTwin uses a Bayesian-style learner model to estimate student mastery and adapt future practice.

The system also uses DeepEval for evaluation and simulated student learning experiments.

CHALLENGES

The main challenges included maintaining accurate source citations, handling OCR errors in handwritten notes, generating meaningful assessment questions, preventing repeated questions, protecting student data, and improving the user interface.

Some limitations remain. Handwritten OCR can produce incorrect text, source-grounded responses sometimes fall back to excerpts, and durable cloud storage is still under verification.

ACCOMPLISHMENTS

I developed and tested a multimodal learning workflow connecting uploaded course materials, source citations, assessments, programming practice, and mastery tracking.

The project also includes a reproducible evaluation pipeline using a team-created dataset and simulated student profiles. Offline metrics are reported as proxy measurements, not proof of real-world learning gains.

WHAT I LEARNED

This project helped me understand retrieval-augmented generation, source verification, Django API development, React frontend integration, learner modelling, software testing, and AI system evaluation.

PROJECT HISTORY AND AI ASSISTANCE

LabTwin began as a classroom and programming-learning project. During the hackathon, I substantially expanded it for Track D with multimodal ingestion, source-grounded tutoring, adaptive assessments, learner modelling, and evaluation.

AI tools, including ChatGPT, assisted with coding, debugging, testing, documentation, and the automated demonstration video.

WHAT'S NEXT

I plan to improve OCR accuracy, assessment question quality, persistent cloud storage, multilingual explanations, and personalization through real-student testing.

GitHub Repository: https://github.com/fjprojects/LabTwin-Track-D-2026

Submitted Development Branch (October 14 version): https://github.com/fjprojects/LabTwin-Track-D-2026/tree/hardening/track-d-demo-20261003