Project description

Echoes helps a caregiver or activity coordinator plan a gentle 30-minute reminiscence session for a person living with dementia. It supports activities, not treatment, and makes no clinical claims.

How Qloo powers Echoes

Qloo is the discovery engine. Echoes resolves the person’s favourites to Qloo entities and uses them as seeds to find related music, TV, film and books from their era and place, with release-year, country and regional music-tag filters, and exclusion IDs after a distress response. Qloo suggests candidates, and local checks decide what reaches the plan. Next week’s plan is built from the caregiver’s recorded responses.

Inspiration

Reminiscence sessions for people living with dementia work best when they are specific: the right music from the years someone was 10 to 25, a TV show they actually watched, a topic that is welcome, and nothing that upsets them. Preparing that takes a caregiver or activity coordinator a lot of time. They have to work out which years matter, find material from the right time and place, check the details, remember the avoid list, and recall what worked last week.

We started this hackathon with Worldlings, a town of AI characters whose cultural tastes drove their decisions. The idea we kept was that taste data can shape real decisions. We moved it to a place where that matters more: helping a caregiver plan a gentle, personal 30-minute session.

What it does

Echoes handles the slow research and checking. The caregiver does the human part.

  1. Enter the basics. The caregiver gives a first name, birth year, where the person grew up, country, language, up to three favourites (artists, films, shows, books), things to avoid, and notes.
  2. Get an era-right plan. Echoes works out the years when the person was about 10 to 25 and uses Qloo to find related music, TV, film and books from that era and place. It builds a timed 30-minute running order with quiet pauses, gentle prompts that always include an easy way out, and invitations built from the caregiver's notes.
  3. Run it at the person's pace. The caregiver can pause, reorder, replace or finish early. Feedback during the session is optional.
  4. Record what happened. In a two-minute wrap-up, the caregiver marks what sparked a response, what wasn't for them, and anything that caused distress.
  5. Next week's plan changes. A "What changed since last time" panel explains why, for example "Because Dorothy responded to X, we added Y".

Every item is labelled You named this or From Qloo. Qloo discoveries are marked "personal familiarity is unconfirmed", and only the caregiver's observations count as facts about the person. A What Qloo added tab shows how many candidates came back, how many passed our checks, and why the rest were held back. There is a print-ready "About me" sheet for care staff.

Echoes supports activities, not treatment. It makes no clinical claims and uses first names only.

How we built it

  • Qloo is the discovery engine. We search for the favourites to get entity IDs, then call insights with those IDs as seeds. Where Qloo supports them, we send a release-year range, a release-country filter for film and TV, and regional music tags found through the tags endpoint. After a distress response we send exclusion IDs so those items don't come back.
  • Local checks decide what reaches the plan. Qloo suggests candidates. Every item then goes through date verification (MusicBrainz for music, TMDB for film and TV), a regional plausibility check, and the avoid list. MusicBrainz runs behind a throttled queue with bounded retries, and successful lookups are cached.
  • A bounded orchestrator calls structured tools: a cultural researcher, a Qloo taste explorer, a session planner and a response adapter. These are real tool calls and branching rules, not four autonomous LLM agents. A kit is capped at 64 tool calls and a 60-second deadline.
  • Memory with provenance. Caregiver-confirmed biography, caregiver-reported observations, and Qloo-suggested discoveries are stored separately, with timestamps. Distress exclusions are always retrieved.
  • Groq disclosure. We use Groq-hosted language models for two things: extracting profile fields from the caregiver's note, and writing session prompts. Groq receives the intake note and the avoid list. It does not receive the first name, birth year, country or feedback text for prompt writing. Qloo receives only item names, entity IDs, country and release-year ranges. Groq is optional, and local fallbacks exist.
  • Stack and hosting. The app is built with Next.js. In demo mode, fictional data stays in the browser. In protected pilot mode, data is encrypted on the server (AES-256-GCM) with expiring, revocable family intake links.
  • Reproducibility. The app has 130 passing tests (as of our last run), a warm-demo script, and an offline check that disables all network calls.

Challenges we ran into

  • Qloo gave us candidates, not guarantees. Artist-level era parameters were accepted but didn't constrain results, so we date-check every track ourselves. Books have no effective region filter, so we use them only as conversation topics.
  • Passing a date check isn't the same as being plausible. Early plans surfaced items that were in-window but wrong for the person, such as an American cop show for a Cuban-born profile and a poetry book for an American one. We added a plausibility gate, so held-back items show their reason instead of silently filling the plan.
  • Qloo's region filter helped but needed care. With a country filter, a Cuban profile moved from US films to four Cuban films. Those films were political, which that profile's avoid list excludes. So every candidate still goes through the avoid-list check.
  • Choosing the right track. Our first version chose the earliest in-window track, which gave us an obscure song and, once, a hymn that could be associated with funerals. We now rank by Qloo notable-song match, then release counts, then wide release, then earliest year.
  • Telling "no coverage" from "couldn't check". A provider outage must never look like an empty catalogue. We separated verified coverage from "Checks still needed", with a retry that only re-checks the failures.
  • A bug we found along the way. One of our free-text fields was never reaching the server. We merged intake and memories into a single notes field.
  • Time and tooling. Our coding-assistant credits ran out partway through, so we worked in small, one-prompt-at-a-time steps, with a frozen demo tag to protect what worked.

Accomplishments that we're proud of

  • A feedback loop you can see: record a response in week one, and week two visibly changes, with a stated reason that traces back to stored data.
  • Honest provenance. Every item says whether the caregiver named it or Qloo suggested it, and nothing from Qloo is presented as a fact about the person.
  • A transparency tab that shows real counts from the plan's own trace, including what we held back and why.
  • A data audit we can state plainly: what Qloo, MusicBrainz, TMDB and Groq each receive and what they never do.
  • Honest failure handling. If we can't verify enough media, the plan says so and falls back to conversation, objects and movement prompts rather than inventing recommendations.
  • A reproducible demo: three fictional profiles (Dorothy, Margaret and Robert), a warm cache of real API responses, and a README that takes a judge from a clean checkout to a running app.

What we learned

  • Qloo works best as a source of candidates with a strong second stage: era, region and avoid-list checks decide what reaches the plan. Putting the constraints into the request helps, but it doesn't replace verification.
  • Plausibility matters as much as accuracy. An item can be correctly dated and still be wrong for the person.
  • For this audience, what we leave out matters as much as what we include. The avoid list, easy ways out and quiet pauses are core to the product.
  • Be exact about what the system is. Calling the pipeline a bounded orchestrator with tool calls, rather than "four AI agents", made it easier to test and explain.
  • Coverage is uneven. Qloo is strongest for Western music and TV, and we say so in the app rather than hiding it.
  • A plan alone is not much value, since an experienced coordinator could sketch one in about ten minutes. The value is in the research and checking behind it, and in what carries over from week to week.

What's next for Echoes

  • Test with real activity coordinators and ask what last-minute session prep actually looks like. We haven't done this yet, and Echoes is not clinically validated.
  • Group sessions for two or three residents, using overlap in what Qloo returns for their favourites.
  • Better regional coverage, including a closer look at which Qloo filters and negative signals can carry more of the era and region constraints.
  • Playable links to specific recordings. Today items link to a search, so the caregiver previews the version and volume first.
  • Production readiness. The current pilot is for one care site with a shared passphrase. Real deployment needs SSO, role-based access, per-user audit logs, managed storage, and organisation-specific review of consent and retention.

Built With

  • nextjs
  • qloo
Share this project:

Updates

Submission history