Inspiration

Dreams Come True grants wishes to children with life-threatening illnesses and has years of photos and videos of those moments. They sit in folders by date with no captions. Finding one photo for a newsletter or a grant report takes staff an hour, and nobody can see which photos have a signed photo release. We wanted staff to find the right moment in seconds, without a single photo of a child leaving their computer.

What it does

Search in plain words: "child hugging a mascot at the theme park" finds matching photos by what is visible, what the caption means and exact keywords.
Search inside videos: videos are split at scene changes and what is said is written down, so search opens a video at the right second.
Three-way pivot: from any photo, show the same event, the same place or the same person (anonymous groups, no names).
Filters: event, person attribute, moment, year and photo release.
Review: staff accept or edit every AI description. Unsure labels are marked with a "?". Questions the AI must not guess, like "is this a family?", are answered by staff.
Curation and safety: favorites, duplicate finder, a bin that empties itself, downloads with a warning when there is no signed release, staff accounts and a review log of every change.

How we built it

Offline, a pipeline describes every photo once. Qwen2.5-VL-3B (4-bit) writes a caption and picks activity, location, clothing, objects and visible moments as JSON. CLIP ViT-B/32 and MiniLM turn each photo and its description into vectors, InsightFace groups faces into anonymous person groups, and Whisper writes down speech in videos. Online, the app (NiceGUI on FastAPI) searches with CLIP, MiniLM and BM25 and merges the three rankings with Reciprocal Rank Fusion. Everything runs on one laptop with an RTX 4070.

Challenges we ran into

Choosing a caption model: we compared BLIP, Florence-2 and Qwen2.5-VL on the same photos. BLIP invented objects and guessed family roles, Florence-2 guessed feelings. Qwen stuck to what is visible.
Speed: Qwen took 34 seconds per photo until we ran it in 4-bit, which brought it to about 5 seconds and 2.6 GB of GPU memory.
Search returned the whole archive for every query until we added a cutoff for photos that do not really match.
Face grouping first chained different people together. Average-linkage clustering fixed it.
Sideways phone photos, iPhone HEIC files, duplicates and Whisper "hearing" words in music.

Accomplishments that we're proud of

A working app that DCT staff could use tomorrow, with privacy built in: no cloud, no names, hidden file names, photo-release checks and a full review log.

What we learned

That the hard part of AI for a nonprofit is not the model but trust: what the AI must not guess, how staff correct it, and how the data stays safe.

What's next for spotlight.io

Run it on the full DCT archive, tune search with the questions staff really ask, link photo releases to DCT's signed forms, and filter spoken names out of video transcripts.