NoteSage

💡 Inspiration

It all started the night before an exam. I wanted to go to the gym, but I also had to study — and my notes looked like an archaeological mystery. Between caffeine, panic, and pre-workout energy, I had a thought:

“What if my notes could just explain this stuff to me?”

That one late-night thought turned into NoteSage — an AI tutor that reads your handwritten notes, understands them, and literally talks back to you like your own personal professor. The goal? Make studying feel less like a chore and more like a conversation.

🧠 What It Does

NoteSage turns handwritten notes into an interactive, voice-powered learning experience. 📝 Upload an image of your notes or a screenshot. 🔍 The app uses OCR (Optical Character Recognition) to extract your handwriting into text. 🤖 Gemini 2.5 Flash analyzes the text, understands the concepts, and generates a clean explanation. 🎙️ ElevenLabs gives that explanation a natural voice, transforming your notes into an AI tutor that literally speaks to you. 💬 You can ask follow-up questions, and the tutor answers — both in text and voice — keeping the learning session conversational. 🔐 Auth0 handles user authentication, so you can log in, save your history, and replay explanations anytime. It’s like ChatGPT, your notebook, and your favorite podcast professor all fused into one.

🛠️ How We Built It

We built NoteSage using Next.js (Pages Router) with a full-stack approach — no extra backend, just API routes: /api/ocr → runs Google Cloud Vision / Tesseract.js for text extraction. /api/explain → calls Gemini 2.5 Flash for contextual, human-like explanations. /api/speak → connects to ElevenLabs to convert text into voice. /api/auth → integrates Auth0 for secure login and user history.

The UI is simple and fast — built with clean CSS (no Tailwind bloat). Users can upload notes, see recognized text, hear explanations, and chat with the AI tutor. We deployed the entire system on Netlify, making it quick to demo and scalable.

⚔️ Challenges We Ran Into

The Gemini SDK and Next.js Turbopack didn’t get along — we ended up using the REST API directly. OCR sometimes couldn’t handle messy handwriting (especially after a long day of note-taking). Getting ElevenLabs audio to sync with real-time responses took fine-tuning. Managing chat context while keeping responses short and natural required prompt engineering finesse.

And, of course, the classic hackathon struggle — balancing code, caffeine, and collapsing sleep schedules.

🏆 Accomplishments That We’re Proud Of

We actually made our notes talk back — and they sound good. Successfully integrated three different AI systems (OCR, Gemini, and ElevenLabs) into one smooth pipeline. Built a fully functional conversational tutor with voice, context, and chat in less than 10 hours. Created something genuinely useful — a tool that could help students everywhere learn faster and smarter.

Managed to still get a workout in somewhere between debugging and deploying. 💪

📚 What We Learned

AI systems work best when you design them like humans: each one doing what it’s good at. Prompt design matters — Gemini gives wildly different results depending on phrasing. ElevenLabs voices make explanations more immersive and easier to focus on. Chaining multiple APIs is like babysitting toddlers — patience, order, and retries are key. And most importantly: you can build something smart and fun even on zero sleep.

🚀 What’s Next for NoteSage

Integrating Gemini Vision to analyze handwritten equations, graphs, and diagrams. Letting users pick different voice personalities for their tutor (chill, hype, or sarcastic). Adding multilingual support, so students can learn in their preferred language. Launching a mobile app for learning on the go — study between sets at the gym. Introducing lesson summaries and quizzes generated from your notes to reinforce learning.

NoteSage: because sometimes your notes need to explain themselves before your professor does.