GPTeach

Inspiration

Traditional lecture notes can be difficult to engage with, especially in complex STEM subjects where understanding concepts often requires more than simply reading through slides. We wanted to combine the accessibility of AI tutors with the familiarity of students' own learning materials, creating an experience that feels like having a personal lecturer guiding you through your notes.

What it does

GPTeach transforms uploaded PDFs and lecture slides into personalised, interactive lessons delivered by an AI tutor.

Using ElevenLabs for realistic voice narration, GPTeach walks students through their material while automatically scrolling to relevant sections and highlighting key concepts. Students can interrupt the lesson at any point, ask questions using their voice, and receive contextual explanations before returning to where they left off.

Rather than simply chatting with a PDF, GPTeach actively teaches its contents.

How we built it

We built GPTeach using Next.js, TypeScript and Tailwind CSS for the frontend, with FastAPI and Python powering the backend.

We used PyMuPDF to extract text and positional information from uploaded PDFs, allowing us to associate concepts with specific locations in the document. An OpenAI LLM processes this information to generate structured lesson segments, including explanations and references to relevant passages.

Our custom PDF viewer, built using PDF.js, interprets these references to automatically scroll and highlight relevant content.

For voice interaction, we integrated ElevenLabs for natural text-to-speech narration and browser-based speech recognition for student questions. A frontend lesson orchestration system coordinates narration, PDF navigation, playback controls and interruptions.

Challenges we ran into

One of our biggest challenges was structuring the backend API endpoints to efficiently handle document processing, lesson generation, and interactive Q&A.

Another major challenge was integrating the different components of our application. We needed to coordinate the frontend PDF viewer, backend LLM, and frontend voice system so that AI-generated explanations, PDF navigation, highlighting, and ElevenLabs narration worked together seamlessly.

Ensuring smooth communication between these independently developed components was particularly challenging under the time constraints of a hackathon.

What we learned

This project taught us that building effective AI applications involves much more than connecting to an LLM API.

We gained experience working with PDF extraction and coordinate systems, structured LLM outputs, speech synthesis and asynchronous frontend state management.

Most importantly, we learned how crucial orchestration is when combining multiple AI services into a seamless user experience. Making different components work together reliably can be just as challenging as implementing the individual components themselves.