PocketTA

PocketTA turns scattered lectures, slides, and PDFs into a personal TA that shows where every answer comes from—and uses your mistakes to guide what you practice next.

Comment

Inspiration

As students, we often find ourselves switching between lecture recordings, PDFs, and presentation slides just to understand a single topic. Sometimes the explanation we need is buried in a recording, while other times it's spread across several documents.

AI tools have made finding explanations easier, but we noticed two limitations. First, it isn't always clear whether an answer is supported by the course material. Second, getting an explanation doesn't necessarily mean we've understood the concept.

We started building PocketTA to address both problems. We wanted a tool that could work directly with a student's course material, provide explanations with verifiable references, and help identify what the student should focus on next.

What it does

PocketTA is an AI-powered study assistant that organizes course material and supports personalized learning.

Students can upload PDFs, PowerPoint presentations, and lecture recordings into a course workspace. PocketTA processes these sources, extracts relevant information, and organizes it into searchable topics. It can also generate structured notes, flashcards, and practice questions.

One of the main features is the course tutor. Students can ask questions, request examples, or compare concepts. Instead of only generating an answer, PocketTA provides references to the original material, including PDF pages, presentation slides, and specific timestamps in recordings. A separate verification step checks whether the cited material supports the generated explanation.

We also implemented an assessment system supporting multiple-choice, short-answer, and numerical questions. Students receive feedback on their responses, along with reports highlighting weak topics and possible misconceptions.

To make subsequent practice more relevant, PocketTA uses Bayesian Knowledge Tracing (BKT) to estimate topic mastery from graded assessments. These estimates help guide future question selection and study recommendations. The system also accounts for insufficient evidence rather than assuming mastery based on limited interaction.

How we built it

We developed the frontend using React and TypeScript, with FastAPI handling the backend. SQLite manages course data, extracted content, assessments, and student learning history.

For document processing, we built an ingestion pipeline that preserves the original location of extracted information. PDFs retain page references, PowerPoint presentations retain slide references, and recordings are processed using FFmpeg and transcribed locally using Whisper. We also integrated vision processing for extracting information from diagrams, scanned documents, and selected video frames.

For information retrieval, we combined SQLite full-text search with semantic search using FastEmbed. We also introduced a summary-based retrieval layer to help locate relevant information across larger course materials.

The generation pipeline supports Cerebras and compatible cloud models, with LM Studio available for local text generation. Rather than relying on a single model response, we separated answer generation, verification, and assessment grading into different stages.

For personalized learning, we implemented a topic-level BKT model that updates its estimates using reliable graded responses. We also maintain assessment history, topic associations, and prerequisite relationships to support future recommendations.

Challenges we ran into

One of our biggest challenges was making generated answers genuinely traceable to their sources. Simply attaching a citation was not enough, since the referenced passage might not actually support the explanation. We addressed this by introducing claim-level verification and checking source versions before saving generated responses.

Processing different types of course material was another challenge. PDFs, presentations, and recordings required different extraction methods, and maintaining accurate references across them was more complicated than we initially expected. Video frame extraction, transcript synchronization, and timestamp navigation required additional debugging and integration testing.

Assessment generation also presented difficulties. Some generated multiple-choice questions contained ambiguous options, making more than one answer potentially valid. We improved question validation and introduced deterministic grading for multiple-choice and numerical questions, while using rubric-based evaluation for short answers.

We also had to handle longer processing operations reliably, including interrupted jobs, retries, cancellation, and progress tracking.

Accomplishments that we're proud of

Our main accomplishment is integrating multiple components into a single learning workflow, from processing course material and answering questions to conducting assessments and maintaining a persistent learning history.

We also built an evaluation pipeline to test retrieval, answer verification, assessment quality, and adaptive learning behavior.

In a small synthetic operating-systems evaluation, our retrieval system found all six expected source locations and correctly rejected two unsupported queries. In a separate live-provider evaluation, the tutor answered all six supported questions and declined both unsupported questions.

Although these results are encouraging, the evaluations use limited synthetic data and do not yet establish how well the system performs across real university courses.

What we learned

Working on PocketTA helped us understand that building an educational AI system involves more than integrating a language model.

We learned the importance of maintaining reliable source references throughout the entire pipeline, designing meaningful evaluation procedures, and handling uncertainty in generated responses.

Implementing Bayesian Knowledge Tracing also introduced us to the challenges of modeling student knowledge. We learned that adaptive recommendations should be based on observable assessment evidence rather than simply assuming that interacting with a topic means understanding it.

Perhaps the most important lesson was that a technically functional learning system is not necessarily an effective one. Measuring actual improvements in student learning requires testing with real users.

What's next

Our next priority is evaluating PocketTA with students using real course material. We want to gather feedback on explanation quality, assessment relevance, and the usefulness of its study recommendations.

We also plan to improve the adaptive learning system using student performance data, refine multimodal content processing, and make the application easier to set up and use.

Our long-term goal is to develop PocketTA into a practical study assistant that helps students make better use of their existing course material and spend more time on the concepts they actually need to work on.

Built With

Share this project:

Updates

Submission history