Full_Video: https://youtu.be/j8IS53CUbWw Only demo: https://www.youtube.com/watch?v=TiTOYB5Lx30
💡 Inspiration
VidPhrase was inspired by a simple problem I kept running into myself. As a beginner video editor, I often had to go through long videos just to find one sentence, one quote, or one specific moment. Unlike text documents, videos do not have a proper Ctrl+F. The problem becomes even worse when the wording is different from what you remember. I wanted to build a tool that makes videos and audio searchable by both exact phrases and meaning.
⚙️ What it does
VidPhrase transforms unstructured video and audio ecosystems into fully indexed, searchable knowledge bases. By ingest-processing YouTube URLs or local media uploads, the platform orchestrates a multi-layered search pipeline across three vectors:
Hybrid Discovery Engine: Performs simultaneous exact-string extraction, typo-tolerant fuzzy matching on transcripts, and full-text indexing of secondary metadata (user comments and video descriptions).
AI-Grounded Semantic Search: Leverages contextual embeddings to understand intent and synonyms, allowing users to find moments based on meaning even if the speaker used entirely different words.
Local Edge Processing: Automates instant localized transcription for private or unindexed audio/video files, completely bypassing the need for expensive third-party cloud API transcription services.
The application dynamically maps results to precise interactive timestamps, generating direct-to-moment playback links and downloadable structured subtitle files for instant navigation.
⚡️ WHAT SETS VIDPHRASE APART (A MARKET FIRST)
A unified video discovery engine of this nature simply does not exist on the market today. There is no single resource, platform, or tool available that combines instant dynamic metadata scraping with deep local and cloud intelligence. Video search remains completely fractured, forcing users to switch between rigid keyword matchers or slow, heavy offline transcription setups. VidPhrase creates an entirely new category through two breakthroughs:
Zero-Storage, Just-in-Time Ingestion: Instead of relying on expensive, heavy databases that require hours of pre-indexing and hoarding video files, VidPhrase operates entirely on the fly. By leveraging yt-dlp in a dynamic architecture, it pulls and parses live metadata instantly. This allows you to perform deep searches on a video uploaded to YouTube just yesterday—or even 5 minutes ago—without background processing delays or infrastructure costs.
The Ultimate Omni-Search Integration: No other platform unifies these four distinct layers into a single query pipeline:
AI-Driven Semantic Search: Interpreting abstract meaning, intent, and synonyms instead of just literal text.
Public Metadata & Comment Search: Capturing real-time audience reactions, crowdsourced corrections, and timestamps.
Literal Keyword & Typo-Tolerant Search: Sub-millisecond syntax matching using advanced algorithms.
Local Edge Processing: Transcribing uploaded offline files instantly using host compute, completely bypassing cloud API fees.
By bridging live web scraping with local AI inference, VidPhrase isn't just an improvement—it is a unique resource addressing a massive void in digital data discovery.
How I built it
I built VidPhrase using Python and Flask for the backend and a web-based interface for the frontend.
🛠️ Tech Stack
| 🏷️ Category | 🚀 Technology |
|---|---|
| 🐍 Backend | Python, Flask |
| 🧠 AI | Google Gemini |
| 🎙️ Speech-to-Text | Faster-Whisper |
| 🎥 Video Processing | yt-dlp |
| 🔍 Search Engine | TheFuzz, AI Semantic Search(Gemini) |
| 📂 File Handling | Werkzeug |
| 🎨 Frontend | HTML, CSS, JavaScript |
I used:
- Flask for the web application and routing
- yt-dlp for extracting YouTube subtitles, descriptions, and comments
- Faster-Whisper for local transcription of uploaded video and audio files
- TheFuzz for fuzzy phrase matching and typo tolerance
- Google Gemini for semantic search, contextual understanding, and synonym detection
- Werkzeug utilities for secure file uploads and processing
I combined these components into a single workflow that can search across multiple sources while keeping the experience simple and intuitive.
🚨 Challenges I ran into
One of the biggest challenges was dealing with real-world language. People rarely remember exact wording. They remember ideas. A phrase may appear as a synonym, another grammatical form, or a completely different expression with the same meaning.
Another challenge was handling different content sources in one application. I had to process YouTube subtitles, comments, uploaded files, and AI-generated semantic results while keeping the experience fast and easy to use.
I also had to balance search accuracy with performance, especially when working with long videos and large transcripts.
🏆 Accomplishments That We're Proud Of
Zero-Cost Edge Transcription: Successfully deployed local, high-speed transcription that matches commercial API accuracy without incurring recurring operational costs.
Unified Intelligence UI: Built a highly intuitive interface that hides immense backend complexity, turning multiple fragmented data sources (comments, transcripts, semantic meanings) into a single, cohesive user experience.
🎓 What I learned
Through building this architecture, we gained deep insights into the nuances of real-time natural language processing (NLP) and data freshness constraints.
First, we learned how human memory favors conceptual recall over literal phrasing, which validated our hybrid search model. Second, we mastered the architectural trade-offs of a zero-storage, just-in-time ingestion pipeline. Bypassing a heavy persistent database meant we had to learn how to perfectly optimize local compute constraints (Faster-Whisper memory footprint) and manage cloud-based API latencies (Gemini) under tight execution windows to ensure instantaneous user query delivery.
What's next for VidPhrase
Next, I want to improve semantic search accuracy, support more languages and media sources, and make navigation even faster.
I also plan to add advanced filters, transcript highlighting, and more powerful ways to explore long-form video and audio content.
My long-term goal is to make VidPhrase feel like Ctrl+F for videos and audio. Made with ❤️, 🐍 Python, and a healthy dose of ☕ caffeine.
Log in or sign up for Devpost to join the conversation.