Mochee

Inspiration

Studying from a PDF usually means reading the same notes over and over. Most AI tutors can explain a topic but don't always teach from the material your professor actually gave you. We wanted to change that.

Our goal was to build a tutor that works directly from the student's own notes. It should be able to point to the exact page it's teaching from and recognize when the material doesn't have enough information to answer a question.

We also wanted studying to feel less like another assignment. That's where Mochee came in.

Instead of staring at a chatbot all day, students get to learn alongside an animated character they can actually interact with. They can work through lessons together and take a break without ever leaving the same virtual room. We wanted to create something that made students genuinely want to come back and study.

What We Learned

One of our biggest lessons was that an AI tutor needs to do more than sound confident. It needs to prove where its answers come from. We learned how to enforce this directly in our code by checking that every cited quote actually exists on the referenced PDF page.

We also learned how much response time affects the experience. What matters most isn't necessarily how quickly the model finishes generating an answer. It's how quickly the student hears Mochee start speaking. That pushed us to focus on streaming audio and making conversations feel more natural.

On the 3D side, we learned how to take a character from Blender into a fully interactive browser environment. We worked through modeling and rigging before creating 27 animation clips and exporting everything into a GLB file that our application could control.

Working as a team taught us just as much. With three people building different parts of the application, we learned the importance of merging changes frequently and making design decisions early. This helped us spend more time improving the project and less time resolving conflicts.

How We Built It

We built Mochee using React 19 and TypeScript with Vite. The entire experience takes place inside a 3D classroom powered by react-three-fiber. This includes Mochee himself and the floating browser windows students use to study.

We created and animated Mochee in Blender. To make his movements feel natural, we developed a state machine that manages animations through priorities and cooldowns. Students can also switch into Play Mode where Rapier physics lets them pick Mochee up and throw him around the room.

We wanted students to have reasons to keep learning beyond simply finishing a lesson. We introduced a coin system and an item shop where students can unlock new interactions. We also built topic-based boss battles that give students a fun way to test their knowledge.

For the learning system, we use pdf.js to process uploaded documents and SQLite FTS5 to index their contents. Every AI-generated answer includes a page reference and a verified quote from the student's notes. The lesson window then highlights that exact passage on the original PDF so students can follow along and see where the information came from.

We also developed a teach-back feature where students explain a concept in their own words. Mochee evaluates their explanation using a rubric generated from the uploaded material. This helps students determine whether they actually understand a topic rather than simply recognizing the correct answer.

The AI runs on our server through the Gemini API with a fallback to a headless Codex CLI session. ElevenLabs powers Mochee's voice while the Web Speech API allows students to ask questions without typing.

Together, these systems create a learning experience where the AI tutor and interactive 3D environment work as one application.

Challenges We Ran Into

One of our earliest challenges was making sure Mochee stayed faithful to the student's notes. Our first version sometimes displayed AI-generated explanations as though they were part of the original document. We realized this defeated the purpose of building a source-grounded tutor.

We redesigned the lesson windows to display the actual PDF and only highlight text that could be verified directly on the page. This gave students a way to check Mochee's answers instead of simply trusting whatever the AI generated.

Voice performance was another major challenge. Long pauses made conversations feel awkward and ElevenLabs credits were limited. We improved the experience by streaming audio as it became available. We also cached generated lines and preloaded upcoming dialogue to reduce delays. To keep usage manageable, we introduced a daily character limit for billed speech.

The animation system also took several attempts to get right. Different actions would sometimes interrupt each other at the wrong moment. Mochee might celebrate a correct quiz answer while still recovering from being thrown across the room. We fixed this by introducing animation priorities and cooldowns so that each action could finish appropriately.

Finally, combining everyone's work was a challenge of its own. Midway through development, we merged three separate branches and made some major design changes. This included replacing our original XP and leveling system with a teammate's coin and shop system. We had to adapt existing features and carry over important fixes without breaking the rest of the application.

Despite these challenges, we came away with something we were genuinely excited to use ourselves.

Mochee started as a simple idea to make studying more engaging. It became a complete learning environment where students can study their own material, ask questions, test their understanding and interact with a character that makes the experience feel more personal.

We wanted to make studying something students look forward to rather than something they have to do. Mochee is our first step toward making that possible.