AI Study Buddy

Turn any webpage into an interactive voice based study session

Comment

AI Study Buddy

Inspiration

I've always learned best when I study with a friend. We explain concepts to each other, ask "why" without hesitation, and keep each other motivated when things get hard.

That kind of learning accelerates understanding, keeps momentum going, and makes studying feel less lonely.

But you don't always have a study partner available — and that's the gap I wanted to solve.

AI Study Buddy brings that collaborative learning experience to anyone, anytime. It's a voice-first AI tutor that helps you learn from any web page through natural conversation, just like studying with a friend who's always ready to help.

What it does

AI Study Buddy is a voice-first learning companion that turns any web page into an interactive study session. Here's how it works:

  1. Paste a URL - Share any web page you want to learn from (articles, documentation, tutorials, research papers)

  2. Select Topics - The app automatically extracts topics from the page structure, and you select up to 8 topics you want to focus on

  3. Start Voice Session - Begin a natural voice conversation with an AI tutor powered by Google Gemini (via ElevenLabs)

  4. Interactive Learning - The tutor teaches topics conversationally, answers your questions, and gives you quizzes to test understanding

  5. Automatic Note-Taking - Notes are saved automatically during the conversation:

    • Summary of what's been covered
    • Q&A from your questions and the tutor's answers
    • Quizzes with your answers, correct answers, and explanations

  6. Track Progress - Completed topics are remembered across sessions, so you can see what you've already mastered and resume where you left off

  7. Download Notes - Export your study notes as a Word document for offline review

The entire experience is voice-first, making it feel like you're having a real conversation with a study partner who's patient, encouraging, and always available.

How I built it

Architecture

Frontend (React + TypeScript)

  • Modern web UI built with React and TypeScript
  • Integrated ElevenLabs Convai widget for voice conversation
  • Real-time notes display with auto-refresh
  • Responsive design with collapsible sidebar
  • Deployed on Vercel for global CDN distribution

Backend (FastAPI + Python)

  • RESTful API built with FastAPI
  • Server-side content extraction from URLs using BeautifulSoup and Readability
  • Session management and progress tracking
  • PostgreSQL database (Cloud SQL) for persistent storage of sessions and notes
  • Word document generation using python-docx
  • Deployed on Google Cloud Run for serverless, auto-scaling infrastructure

AI & Voice

  • ElevenLabs Agents platform for voice conversation (speech-to-text and text-to-speech)
  • Google Gemini (configured via ElevenLabs) for LLM reasoning and conversation intelligence
  • Webhook tools that connect the agent to the backend API
  • Agent automatically saves notes, tracks progress, and adapts to user understanding

Key Technical Decisions

  • Voice-First Design: Chose ElevenLabs for high-quality, natural-sounding voice conversations
  • Server-Side Content Extraction: URLs are fetched server-side for security and reliability
  • Session-Based Architecture: Each study session has a unique ID, allowing multiple sessions per URL with different topic selections
  • Progress Persistence: Completed topics are tracked across sessions using PostgreSQL for reliable data storage
  • Google Cloud Integration: Leveraged Cloud Run for backend hosting and Cloud SQL for database persistence

Integration Flow

  1. User pastes URL → Frontend calls /topics endpoint → Backend extracts topics
  2. User selects topics and starts session → Frontend calls /sessions/start → Backend creates session
  3. User starts voice call → ElevenLabs widget connects to agent
  4. Agent uses webhook tools to:
    • Fetch page content via /extract
    • Get session info via /sessions/latest
    • Save notes via /notes/set_summary, /notes/append_qa, /notes/append_quiz
    • Mark topics as done via /sessions/progress/mark_topic_done
  5. Frontend auto-refreshes notes every 4 seconds during active sessions
  6. User can download notes as Word document via /notes/download.docx

Challenges I ran into

  1. Content Extraction Reliability: Different websites have vastly different HTML structures. I had to handle edge cases like GitHub markdown, pages without clear headings, and various content formats. I solved this with a combination of Readability for main content extraction and BeautifulSoup for heading structure analysis.

  2. Real-Time Notes Synchronization: Keeping notes in sync between the agent (which saves them) and the frontend (which displays them) required careful polling logic and state management. I implemented auto-refresh that updates notes every 4 seconds during active sessions.

  3. Voice UI/UX Clarity: Initially, the user flow wasn't clear - users weren't sure what to do after starting a session. I iterated on the UI to add clear step-by-step instructions and visual feedback, making the flow more intuitive.

  4. Progress Tracking Complexity: I needed to distinguish between "selected topics" (fresh selection for a session) and "completed topics" (aggregated across all sessions for a URL). This required careful database schema design and UI logic to show progress appropriately.

  5. Session State Management: Managing multiple sessions, ensuring the right session is loaded, and handling edge cases like deleted sessions required robust state management in the frontend.

Accomplishments that we're proud of

  1. Seamless Voice Experience: Created a natural, conversational learning experience that feels like talking to a real study partner, not a chatbot

  2. Intelligent Progress Tracking: Built a system that remembers what you've learned across sessions, so you can see your progress and avoid repeating topics you've already mastered

  3. Automatic Note Generation: The AI tutor automatically saves comprehensive notes (summaries, Q&A, quizzes) without requiring any manual note-taking from the user

  4. Flexible Learning: Works with any web page URL, making it easy to learn from any source - documentation, articles, tutorials, research papers, etc.

  5. Production-Ready Infrastructure: Deployed on Google Cloud Run and Cloud SQL with proper database persistence, making the app reliable and scalable

  6. Clean, Intuitive UI: Built a responsive interface that adapts to different screen sizes and provides clear guidance at each step of the learning journey

  7. Integration Excellence: Successfully integrated multiple services (ElevenLabs, Google Gemini, Cloud Run, Cloud SQL) into a cohesive, working application

What I learned

  1. Voice AI Capabilities: Explored the power of voice-first interfaces and how natural conversation can enhance learning compared to traditional reading/writing interfaces

  2. Agent Architecture: Learned how to configure AI agents (ElevenLabs with Gemini) and connect them to backend services via webhook tools for a complete application

  3. Content Extraction Challenges: Gained experience handling the messy reality of web content - no two websites are structured the same, requiring flexible parsing strategies

  4. State Management Complexity: Discovered the intricacies of managing application state when dealing with real-time updates, multiple data sources, and user interactions

  5. Google Cloud Services: Deepened understanding of Cloud Run for serverless deployment and Cloud SQL for managed PostgreSQL databases

  6. User Experience Iteration: Learned the importance of iterative UI/UX refinement - what seems obvious to developers isn't always clear to users, requiring constant feedback and improvement

  7. Session-Based Design: Understood how to design systems around session-based workflows, balancing flexibility (multiple sessions per URL) with simplicity (easy to resume where you left off)

What's next for AI Study Buddy

  1. Multi-Language Support: Add support for learning content in multiple languages, with the AI tutor adapting to the user's preferred language

  2. Study Groups: Allow users to share sessions with friends, enabling collaborative learning where multiple people can study together

  3. Spaced Repetition: Implement intelligent review scheduling that reminds users to revisit topics at optimal intervals for long-term retention

  4. Voice Profiles: Let users customize the AI tutor's voice, speaking style, and teaching approach to match their learning preferences

  5. Advanced Analytics: Add detailed analytics showing learning progress, time spent on topics, quiz performance, and personalized insights

  6. Mobile Apps: Expand beyond web to native iOS and Android apps for on-the-go learning

  7. Integration with Learning Platforms: Connect with popular learning platforms (Coursera, Khan Academy, etc.) to import course content directly

  8. AI-Powered Study Plans: Generate personalized study plans based on user goals, deadlines, and learning history

  9. Community Features: Build a community where users can share study sessions, recommended content, and learning tips

  10. Enhanced Note Formats: Support for more note formats (PDF, Markdown, Notion integration) and better note organization features

The vision is to make AI Study Buddy the go-to companion for anyone who wants to learn effectively, turning the solitary act of studying into an engaging, conversational experience that accelerates understanding and keeps motivation high.

Built With

Share this project:

Updates