WhatTheFilm!

• 

  WhatTheFilm

Inspiration

We wanted a Shazam for movies and TV. Whenever we saw a funny meme, a random screenshot, or a clip out of context, we wished there was a way to instantly identify the source. That inspired WhatTheFilm!

What it does

WhatTheFilm! lets users upload any screenshot from a movie or sitcom, and the system predicts the correct title with confidence scores. It helps users discover new shows, settle debates, and reconnect with old favorites.

How we built it

We trained our system using the CoCoN (Contrastive Contextual Network) model for image representation.

Data Collection: Gathered screenshots from a wide range of movies and sitcoms.

Modeling: Fine-tuned CoCoN to generate embeddings that capture contextual and visual cues.

Similarity Search: Used cosine similarity to compare embeddings with a pre-built database.

Frontend: Built a simple web app where users can upload images and instantly see predictions.

Challenges we ran into

Collecting diverse and well-labeled datasets.

Preventing the model from overfitting to costumes or backgrounds.

Optimizing inference speed for real-time results.

Handling visually similar sitcoms with nearly identical sets.

Accomplishments that we're proud of

Successfully fine-tuned CoCoN to achieve strong accuracy on our dataset.

Built a fully working prototype with fast predictions.

Created a fun and engaging user experience that feels intuitive.

What we learned

How powerful CoCoN is at capturing nuanced visual context.

The importance of good data hygiene and balanced sampling.

That combining model accuracy with speed is key to usability.

What's next for WhatTheFilm!

Expanding the dataset to include more global cinema and anime.

Adding scene and season-level predictions for sitcoms.

Building a browser extension to identify content directly from memes and clips.

Exploring multimodal approaches by integrating subtitles or audio alongside frames.

Built With

• cocon
• datasets
• image
• scaled-recognition