Inspiration

Students often leave a lecture having understood about half of it, with no one nearby they know well enough to ask. They reread their notes alone, and rereading feels like learning but isn't. Making friends at a big school is hard, studying alone is isolating, and teachers can't see who is struggling until an exam exposes it. We wanted a tool students would open because they want to: one that explains the material back to them in a way that clicks and introduces them to people studying the same thing.

What it does

Fairytails turns a student's notes, or a teacher's class material, into a pixel-art adventure. Students upload their own notes or join a class with a six-character code, then pick a Sports or Fantasy world, which acts as their interest and shapes how everything is explained. If a student didn't follow their professor, a "Hear this lesson" button has an AI coach explain the concepts using that world's analogies and the lesson's actual facts, with a readable transcript alongside. A solo round then tests them with six spoken encounters and three lives, and missed concepts come back more often. Afterward, the app matches each student with study partners: classmates who picked the same world come first, and independent students are also matched by subject similarity, so they meet strangers working on the same material without any cold messages. Matched students form a party of two to four and take on a shared boss built from the group's weakest concepts. Each player explains a concept by voice or text, an AI rubric grades the explanation, and everyone must pass for the team to win, so the players need each other. Afterward, they can add party members as friends and chat. Teachers get a dashboard showing who is ready, which parties formed, and which topics need review. For investors, understanding plus belonging is what draws students in, the party mechanic gives each student a reason to invite classmates, and the teacher dashboard is the paid hook for schools.

How we built it

The frontend uses React, TypeScript and Vite, with local pixel-art sprites. The backend is Express on Node with Postgres on Tiger Data, and pgvector handles subject embeddings and matching. Gemini generates the game packs, embeddings, boss grading and coach audio, and Meta's Muse handles voice transcription and coach scripts. Recordings are converted to 16 kHz mono WAV in the browser before they reach the grader. The server controls turns, rubrics and pass/fail, which keeps the boss fight fair, and friends and chat run through the same member accounts. We added a fallback API key for quota failures and a fully offline demo.

Challenges we ran into

Latency came first, since generating a game and its voices takes time, so voice generation now starts while the game is still being built. Voice grading was harder than expected because browser audio had to be converted into a format the model accepts, along with microphone permissions and cleanup. Multiplayer state was another challenge, since matching and the boss fight need consistent state across players, which we solved with server-owned turns and polling. Matching also took tuning, because we had to measure how similar two students' notes really are before deciding who counts as a good partner. Finally, quota limits and missing keys could not be allowed to break a live demo.

Accomplishments that we're proud of

We built the full loop of notes, AI explanation, game, match, party, boss and friends, and it works for solo students and whole classrooms. Spoken explanations are graded by AI, so the game tests understanding rather than recognition. The offline demo runs with no backend, keys or database, and the teacher dashboard turns student play into data teachers can act on.

What we learned

Game design matters as much as the AI, since lives, retries and shared stakes are what keep students going. Grading an explanation works better than grading a multiple-choice answer. The social side matters as much as the study side too, because a boss that can only be beaten together is a better reason to meet someone than a study-group sign-up sheet.

What's next for Fairytails

Next we want to pilot with a few classrooms and campuses and measure whether students make connections, keep coming back, and learn more. We plan to add more worlds with themes students vote on, and integrations with tools like Canvas and Google Classroom. We plan to sell to schools, districts and universities through per-classroom and per-school licensing, with a free tier for independent students, and we also want to add progress tracking and parent reports.