ClipStudy.ai

A video search engine to learn fast through videos.

Comment

Inspiration

Many students struggle to efficiently review specific concepts in long lecture recordings. Even with YouTube’s auto-generated chapter labels, users still have to manually scroll through these sections, and these labels often miss hyperspecific or nuanced topics, making it hard to quickly access exactly what’s needed.

What it does

ClipStudy allows users to paste in a YouTube video link and use natural language to ask for the specific concept they want to review. Our backend then analyzes the video transcript, finds the relevant timestamps, and automatically jumps to that point in our embedded video.

How we built it

We built ClipStudy.ai by developing a local script to fetch and clean YouTube video transcripts using yt-dlp and custom SRT parsing logic. Once the transcript is processed, we leverage Claude 4 Sonnet to analyze the transcript and identify segments relevant to a user’s query, using a prompt-based approach for granular concept detection. The backend coordinates transcript extraction and segment analysis, while the frontend presents an interface for users to submit links and review the identified video segments. All intermediate data (transcripts and segment metadata) are managed locally for efficiency and privacy.

Challenges we ran into

  • Curating and cleaning diverse YouTube transcripts with varying quality and formats
  • Fine-tuning prompt engineering with Claude 4 Sonnet to accurately detect nuanced educational concepts
  • Balancing fast response times with thorough transcript analysis for a smooth user experience

Accomplishments that we're proud of

  • Successfully automated the end-to-end process from transcript extraction to concept-based video navigation
  • Developed a robust transcript cleaning pipeline that handles a wide range of YouTube lecture formats
  • Achieved accurate, AI-powered detection of nuanced concepts and seamless video segment jumping
  • Built a privacy-focused system that processes and stores all user data locally

What we learned

  • The power of large language models for granular concept detection in long-form educational videos
  • The value of prompt engineering for improving AI accuracy and relevance

What's next for ClipStudy.ai

  • Expand support to additional video platforms beyond YouTube, such as Vimeo, Zoom recordings, and educational portals.
  • Integrate with popular notetaking and productivity apps such as Notion.
  • Add support for YouTube playlists and video-searching inside the system (backend code exists).

Built With

Share this project:

Updates