Video Index

• 

  Processing videos using FAISS-CPU to create vector clusters

• 

  Videos clustered and embeddings created using FAISS-CPU + ResNet50

• 

  Similarity Search on Cat video - Most similar found, closest one also found.

• 

  Similarity Search on Dog video - Most similar found, closest one also found.

Video Index: AI-Powered Visual Video Search Searching large video libraries is difficult because most tools rely on filenames and metadata, not the visual content itself. We built Video Index, an AI-powered system that lets you search and organize your entire video library based on what's in the videos.

Key Features Visual Similarity Search: Find videos that look alike. Our system uses ResNet-50 to analyze video frames and find a match, regardless of the filename.

Automatic Clustering: Automatically groups your library into visual categories (using K-Means, DBSCAN, etc.) so you can discover related content.

Real-time Indexing: A file watcher automatically processes new videos as soon as you upload them.

Web Interface: A simple React UI to upload, search, and browse your videos.

Fast Search: Uses FAISS to deliver sub-second search results, even with many videos.

Technology Stack Backend API: Node.js (Express)

ML Pipeline: Python

Video Analysis: PyTorch & ResNet-50

Clustering: scikit-learn

Similarity Search: FAISS

Frontend: React & Tailwind CSS

Database: SQLite

Integration: The Node.js server orchestrates the pipeline by calling Python scripts as subprocesses to handle the heavy ML tasks.

Challenges & Accomplishments We're proud of building a fully functional, end-to-end pipeline that handles everything from upload to search.

The biggest challenge was integrating Node.js and Python. We had to solve complex issues with cross-platform command execution (especially on Windows), file path management, and ensuring our processing pipeline was consistent for both indexing and querying.

We also solved performance bottlenecks. Our initial "rebuild-on-upload" system was too slow, so we implemented a "debounced" file watcher that waits for a few seconds of inactivity before re-indexing.

What We Learned FAISS is fast: It's an incredibly powerful tool for high-speed vector search.

Hybrid Stacks are Hard (but Worth It): Bridging a Node.js API with a Python ML backend is complex but allows you to use the best tool for each job.

PCA is Key: Using PCA to reduce embedding dimensions was crucial for cutting down noise and speeding up the clustering process.

What's Next Action Recognition: Integrate temporal models (like TimeSformer) to understand actions in videos, not just static frames.

Multi-modal Search: Add audio analysis to search by both sound and visuals.

Text-to-Video Search: Implement a model like CLIP to allow searching with natural language text.

Scale: Migrate from a file-based index to a dedicated vector database like Milvus or Pinecone.

Built With

• axios
• css3
• express.js
• faiss
• html5
• javascript
• matplot
• node.js
• numpy
• python
• pytorch
• scikit-learn
• tdqm
• torchvision