MediaID AI

• 

• 

• 

Inspiration

MediaID: Project Story

The inspiration behind MediaID came from the way people instantly recognize songs using apps like Shazam, yet there is still no seamless experience that combines music recognition and movie scene recognition into one intelligent platform.

We wanted to create an application where users could simply: Play a song near their phone Hum a melody Record a short movie clip Or capture a scene on screen …and instantly discover what media they are interacting with.

The vision was to build a “universal media identifier” , an AI-powered assistant capable of understanding both sound and visuals in real time.

What it does

MediaID is an AI-powered multimedia recognition application that identifies: Songs from microphone audio Movies from short video clips Background music from noisy environments Film scenes using computer vision and audio analysis

The application works similarly to Google Sound Search and Shazam, but extends beyond audio by also supporting movie recognition through visual analysis. Users simply tap the recognition button, grant microphone or camera access, and the system processes the captured media to provide accurate results within seconds.

How we built it

We designed the system using a hybrid AI architecture composed of: Audio Recognition Pipeline

For song recognition, we implemented: Audio recording through microphone access Spectrogram generation FFT (Fast Fourier Transform) analysis Acoustic fingerprint extraction Hash-based fingerprint matching

This approach was inspired by how professional music recognition systems work internally.

Video Recognition Pipeline

For movie detection, we used: Video frame extraction Scene analysis OCR text detection Audio extraction from video AI-based similarity matching

The system combines audio and visual clues to improve recognition accuracy.

AI & Matching System

We implemented: Vector embeddings for similarity search Fingerprint-based exact matching Confidence ranking algorithms Noise reduction and normalization

The recognition engine prioritizes fingerprint matching first, then uses AI similarity search as a fallback for uncertain matches.

Challenges we faced

One of the biggest challenges was recognition accuracy.

Initially, our system relied heavily on AI similarity embeddings, which caused incorrect song matches because songs with similar rhythms or genres were being confused with one another.

To solve this, we redesigned the architecture to use: Acoustic fingerprinting as the primary matching method AI embeddings only as a fallback Confidence thresholds to avoid false positives

Another challenge was handling: Background noise Low-quality recordings Different microphone hardware Short audio clips

We improved results using noise filtering, longer sample durations, and optimized frequency analysis.

What we learned

Through building MediaID, we learned: How audio fingerprinting systems like Shazam operate The importance of spectrogram analysis in recognition systems How AI embeddings differ from exact fingerprint matching How multimodal AI systems combine audio and visual intelligence Real-world optimization techniques for mobile AI applications

We also gained experience designing scalable AI pipelines suitable for real-time mobile applications.

Future plans

In the future, we plan to add: Hum-to-search functionality Offline recognition mode Live concert detection TV show recognition Personalized recommendation engine Cloud synchronization Social sharing and trending discoveries

Our long-term goal is to make MediaID the ultimate intelligent media discovery platform.

Built With

• ai
• analysis
• audio
• chromaprint
• computer
• embeddings
• faiss
• fastapi
• fft
• firebase
• flutter
• javascript
• learning
• librosa
• machine
• native
• node.js
• opencv
• processing
• python
• react
• spectrogram
• sqlite
• tensorflow