Inspiration
Everyone has had a song, film or website on the tip of their tongue: you remember the lyric, the scene, your grandfather’s turntable, but not the name. These descriptions are long and vague, it isn’t clear which detail will lead to the answer, and not every detail is reliable. Search engines treat every word as a filter, so the most vivid detail often floods the results with the wrong things. On the BLUR benchmark, even strong agents struggle with these questions. We wanted an agent that searches the way a patient friend would: try the obvious first, and when that fails, work out which part of the memory is holding the search back.
What it does
Adaptive Tip-of-the-Tongue Search takes a free-text description and returns the specific item.
- Stage 1: the agent searches the whole description with Telem, choosing 1–2 providers and optional domains per search, and answers with a confidence.
- Confidence gate: at 0.95 or above it stops. Below that, it looks closer.
- Stage 2: it turns the memory into weighted conditions (how specific × how sure the user sounds; “I think…” lowers a weight) and checks each against what was found. The most-trusted condition that search can’t confirm is relaxed: it stops filtering the search, and the agent searches the remaining conditions and verifies with Telem search and page reads.
Example: “a Nigerian highlife song… about a woman who loves to rub pancake… ‘lika this lika that, oh baby’”. Today’s agent searches it all at once and answers Flavour. Ours relaxes the unconfirmed lyric, searches pancake + highlife, and finds “Baby Pan Cake” by B.E. Batta & His Eastern Star Dance Band (1960s). Reading the song’s page with Telem fetch afterwards showed the lyric was real all along, just not searchable.
How we built it
- Telem (SearchHack deployment) for all retrieval: batched multi-query searches, agent-chosen providers (Brave, Exa, Tavily, You, Parallel, TinyFish) and domains, published-date filters, and
fetchto read whole pages. Every call is cached on disk with a cost estimate and session id. - Gemini 3.5 Flash Lite (free tier) as the agent, through native function calling in a hand-written loop with no framework, so every step is logged.
- Soft-AND ranking (a weighted p-norm) to score candidates against conditions, so one weak condition lowers a score instead of excluding the answer.
- Evaluation on BLUR’s labelled validation split with an LLM judge (Gemini 3.1 Flash Lite), plus a resumable experiment runner and a step-by-step replay viewer built from the saved traces.
Challenges we ran into
- Day-one setup: our Telem key returned 401 until we found the hackathon deployment’s own API base URL.
- Free-tier limits: 500 agent calls a day meant a fixed set of 18 hard questions and one run per configuration. BLUR’s public test split has no answers, so we used the validation split.
- Auto-routing on vague queries: Telem’s router picked 2 providers for only 17 of 136 queries (programming and banking topics). Everything else went to all providers, costing 3–4× for the same accuracy, so we let the agent choose providers instead.
- Correct results we never read: in our example, Telem retrieved the right record, but it landed at #20 while our tool showed the model only the top 15.
- Hedged doesn’t mean wrong: users hedged on details that turned out to be true, and stated untrue or unsearchable details confidently. Down-weighting by tone alone was not enough.
Accomplishments that we're proud of
- 14/18 on hard BLUR questions, versus 12/18 for today’s single-pass agent and 13/18 for simply retrying when unsure, at about $0.08 of Telem search per question.
- A provider head-to-head on identical queries for tip-of-the-tongue search: Exa 31%, Brave 29%, Tavily 27%, down to Ceramic 0%. Our agent’s own picks concentrated on the top three.
- A confidence gate that actually separates right from wrong: answers at ≥0.95 were right 17/19 times, below it only 2/9.
- A replay viewer and honest write-ups of what didn’t work, not only what did.
What we learned
- Search first, decompose only when unsure. Splitting every query into clues doubled the cost on easy questions without improving accuracy.
- “Unconfirmed” is not “false.” The best move is to stop filtering on a detail search can’t confirm, not to declare it wrong.
- Reading matters as much as retrieving. How results are ordered and cut before the model sees them decided cases.
- The remaining failures are mostly model knowledge: a small model never connected “a logo with a letter and a punctuation mark” to Y! (Yahoo Finance).
What's next for Adaptive Tip-of-the-Tongue Search
- Show the model results round-robin across queries and providers, so a correct result can’t sit unread at #20.
- Make the “which condition can’t be confirmed” check evidence-based and reliable, and turn abstract clues into concrete names before searching.
- Search remembered lines as lyrics or quotes, and let the agent read pages with Telem fetch on its own.
- Evaluate on fresh questions with multiple runs and a stronger model, and test Telem routing with topic hints.