🥇 Knots: The Best Recommender Yet

Your research and your entertainment, tied together, with receipts.

Ask Alexa+ "what are we doing today?" and Knots tells you what you've been circling. Then it recommends films, series, documentaries, explainers and papers, and every pick comes with the reason it was chosen and the evidence behind it.

Code: github.com/spacey-cadet/knots

Backstory: twenty seconds that taught me more than a shelf of books

I'm at a stage of life where I'm deeply curious about the nature of reality. I've read Buddhist philosophy, Zen mind, The Body Keeps the Score, Branching Streams, scientific takes on therapy, and even papers on the mathematical foundations of game theory. I've studied music theory and AI, and the more I learned, the more everything turned out to be connected.

Then I watched Dark, and one scene did what none of those books had.

Old Helge is in search of his younder self(literrally) and he want to prvent him from commiting a crime. He keeps chanting to himself "He has to stop... He has to stop.". And then soemthing shifts. As if he had become enlightened, he said finally*"I have to stop"*. It felt like the series did a mic drop here :)

I had never pictured a future version of me deciding my present. That scene made it concrete. Reifying your future self and letting them weigh in on your decisions is a real technique, and twenty seconds of cinema made me feel it in a way pages of theory never did. Pantheon did the same for an AI future, and music theory showed me the mathematics hiding underneath sound.

Research is already hard. Abstract ideas like time, recursion and free will resist reading alone. Cinema gives them an emotional and visual handle. Research gives them structure and grounding. I stopped believing that what I watch for fun and what I pursue intellectually are separate things, and walling them off wastes the appeal of entertainment and the substance of research.

I'm also a cinephile, and I don't think entertainment should be random. It's emotionally charged, so you should be in control of what you consume, and you can use that charge to your advantage.

Knots: the solution

Who it's for: researchers and cinephiles, moreso people who are both. If you think hard about abstract ideas and love film, Knots finds the films that make those ideas concrete.

Why it's a major improvement: today's recommenders are loose. They optimize for popularity and for what people like you clicked, and they know nothing about what you're trying to understand. Knots tightens them, which is where the name comes from.

Context at the center, not popularity. Popularity is not a ranking signal. Quality is only a floor, and fit to what you actually said decides the order.
Concepts, not keywords. It extracts the ideas you're working on, plus unstated themes your words imply, and it withholds any theme it can't trust.
A note that earns the pick. Each pick comes with a short note on why you would love it, tied to your own words and the idea behind it. That note is where the convincing happens.
Evidence behind the scenes. Underneath every card is an internal evidence ledger (matched catalog tags, the item's own summary, your own sentence, paper status). A card with nothing to stand on is never shown, and every number, name and quote in a note must trace back to that evidence or the note is thrown away.
Saying less beats guessing. If nothing clears the quality bar, Knots says so and asks what you'd like to dig into. It never invents a title.
Watch-and-learn pairs. A drama plus an explainer or paper on the same idea, as a double feature.

Technical details

Architecture

session text ─► understanding ─► retrieval ─► quality floor ─► fit ranker ─► plan
(Alexa+)+memory  Bedrock: topic,   TMDB films/series/docs,   learned       6-feature    watch / read /
                 concepts,         YouTube explainers,        similarity   logistic     explore, pairs,
                 themes, mediums   arXiv papers               floor        model        receipts, spoken line

Knots is a real MCP server (Python, FastMCP, Streamable HTTP) with an Alexa custom-skill endpoint beside it. It exposes knots_partner, rate_pick, get_recommendations, extract_interests, search_content, what_do_you_remember and forget_me.

Why not a lazy recommender

The famous techniques (SVD, neural collaborative filtering, user-based filtering) need a utility matrix of many users' ratings. A personal assistant has one user and no matrix, so it lives in permanent cold start. And "most popular" is exactly the randomness I wanted to remove.

So Knots is a hybrid, content-based system:

an LLM (Amazon Bedrock) for understanding what you're circling,
embeddings (Cohere Embed v4) for meaning,
small models trained on my own labels for ranking and trust.

Collaborative filtering comes later, once there are enough users to build a utility matrix.

Understanding: concepts, not keywords

Bedrock's Converse API, with a forced tool call, returns structured JSON per conversation: the surface topic, the abstract concepts at stake, themes, film-tag keywords and the best formats. Model output is treated as untrusted (validated, clamped, session text fenced in tags). If anything fails, a heuristic extractor takes over, and the response reports which one ran.

TMDB is queried by keyword, not title, so "free will" finds films tagged with it, not films named "Free Will".

Labelling

A personal assistant has no crowd to learn from, so I used labels to stand in for one. I had an LLM generate 30 conversational sessions (what a curious person might say to Alexa), ran them through Knots, and labelled every suggestion good or bad myself, with a note on why. The point was to improve Knots' general judgement (what a good pick and a trustworthy theme look like), not to model a particular user. Personalization comes later, at runtime, from context.

Picks (270): good or not, with a reason when not (relevance, quality, repeat).
Themes (101): supported by what the person said or not, with an error type (over-psychologizing, terminology overreach, narrative projection, invented "X vs Y").
Pairs:

How the labels were made. The sessions were written by an LLM. The good/bad verdicts are mine. I used an LLM to help write the notes for sessions 1 to 24 and wrote sessions 25 to 30 by hand. Sessions 25 to 30 are held out and never trained or tuned on. The limits are real: one labeller, synthetic sessions, a narrow range of topics, and LLM-assisted notes that also feed the guidance given to the extractor.

Training: three small, honest models

Nothing here is a black box, and nothing is user-specific. Each part is a small model that learns general judgement from my labels. It is small enough to inspect and is scored on sessions it never saw.

Theme judge. A feature critic plus nearest-labelled-themes, averaged and Platt-calibrated. Leave-one-session-out AUC is 0.84 [0.77, 0.91]. Keeping themes at calibrated $ p \ge 0.5 $ keeps 36 of 101 themes at 75% supported, against a 44% base rate.
Fit ranker. A six-feature logistic model that scores each candidate:

$$ \text{fit}(c) = \sigma\big(\mathbf{w}\cdot\mathbf{x}^{(c)} + b\big), \qquad \mathbf{x}^{(c)} = \big[\text{sim-session},\ \text{sim-concept},\ \text{overlap},\ \text{quality},\ \text{medium},\ \text{authority}\big] $$

Features are z-scored within each candidate set, so the weights don't depend on which embedder ran. Leave-session-out AUC is 0.69, against 0.66 for my hand-set priors and 0.32 for the old tag-overlap score. On holdout the AUC is 0.80 (n=55, wide interval).

Quality floor. A similarity cut-off chosen as the most lenient one whose surviving picks were right at least 75% of the time. On holdout that gave 83% precision while keeping 44% of picks (only 24 picks, so I treat it as a small sample).

Similarity is cosine similarity between embedding vectors:

$$ \text{sim}(q, d) = \frac{q \cdot d}{\lVert q \rVert \, \lVert d \rVert} $$

Pairs

A watch-and-learn pair takes one thing to watch ($ W $) and one to learn from ($ L $) about the same interest:

$$ \text{pair}(W, L) = \min\big(\text{fit}(W),\ \text{fit}(L)\big) \cdot \cos(W, L) \cdot \text{level}(W, L) $$

The min stops a strong half rescuing a weak one, cos requires them to cover the same idea, and level follows a ladder (drama, then explainer, then survey paper). The floors and the ladder are untuned design choices, which is why pairs are the next thing I label.

Metrics: beyond accuracy

Accuracy alone can reward a recommender for showing the obvious. So I evaluate on the top 3 picks with session-resampled 90% intervals, against random and popularity baselines, and I report accuracy next to the things it can hide.

$$ P@k = \frac{|\text{relevant} \cap \text{top-}k|}{k} $$

The headline metric is grounding, where $ E $ is the number of independent evidence types behind a pick (a matched TMDB keyword, a quoted theme, overview similarity):

$$ \text{Grounding@}k = \frac{1}{k}\sum_{i=1}^{k} \text{good}_i \cdot \text{supported}_i \cdot \frac{\min(E_i, 3)}{3} $$

On the Bedrock extractor (24 train sessions, top 3):

P@3: 0.76 (random: 0.74, so accuracy barely separates rankers).
Grounding@3: 0.46 (random: 0.39).
Overreach rate: 0.19 (random: 0.22). This is the share of picks labelled not supported, and it acts as a gate: a retrain is refused if overreach rises.
Creative and grounded (strict): 0.19. Picks that are good, supported by two or more evidence types, and far from your own words.
Theme coverage: 0.79. Supported themes served by a good pick.
Interest coverage: 0.76. Your interests that at least one pick answers.
Diversity (ILD, Cohere): 0.68. Mean pairwise cosine distance among the top 3.
Novelty 0.52, unexpectedness 0.48, serendipity 0.35. Rarity, distance from your words, and good picks that are also unexpected.
Coverage: 65 distinct items in 69 slots (94% unique, Gini 0.06). There is no catalog denominator, so this measures repetition, not reach.

I removed an earlier "creativity" metric because it rewarded far-fetched picks. Distance now earns nothing on its own. A bad or unsupported pick scores zero however far it is.

The biggest single effect in the whole system is the extractor: Bedrock reaches P@3 0.76 and Grounding 0.46, against 0.44 and 0.10 for the weak heuristic fallback.

Online metrics. Every response logs which picks were shown, and rate_pick and the skill's "good pick" and "not for me" log a verdict. A monitor reports the positive rate, an exploration gap, score drift, empty-list rate and a filter-bubble alarm. Online signals never become training labels, because only explicit verdicts do. A retrain ships only if it beats the champion on the frozen holdout and overreach doesn't rise.

Context awareness

What context is for. Knots has to answer two different questions: what are you trying to understand right now, and what do you keep coming back to? Context is how it tells a passing question from a real interest.

How it's used.

Per-conversation understanding. Each conversation is extracted on its own, in parallel and cached. Mixing them into one prompt blurs unrelated topics, and cost grows with history.
Two signals merged into threads. The newest conversation means now (boosted, and its strongest thread always stays in the top two). Topics that recur across several conversations mean enduring. A thread's weight halves every 14 days:

$$ w = 0.5^{\,\text{age}/14} \cdot \text{confidence} \cdot \text{boost} $$

Your own words drive the ranking. Similarity to what you actually said is the largest learned weight (0.77). Similarity to the model's paraphrase of it got a slightly negative weight (−0.26). With only 24 training sessions I wouldn't over-read that, but your wording beat the model's summary of it. Context becomes the note. The note speaks to your own words and the idea behind them, and the spoken line says things like "you've come back to X across N conversations."
Context picks the format. Casual vs deep interest and the ranked mediums decide whether the lead pick is a film, an explainer or a paper.

The outcome. The score without your words (tag overlap alone) did worse than chance, with AUC 0.32. With your words in the ranker it reaches 0.69 leave-session-out and 0.80 on holdout. On Bedrock lists, ordering by session fit gave P@3 0.80 against 0.76 for tag overlap, a modest lift with overlapping intervals. Memory recall and the rolling window are covered by tests (a related past conversation is recalled, an unrelated one isn't, nothing is double counted, memory holds no raw text, and a memory failure never breaks an answer). Their effect on pick quality is not yet measured, because my 30 evaluation sessions are single conversations.

Minimal statefulness

Knots stays small on purpose, with a rolling window: the last 8 conversations within 60 days, each trimmed to its latest 3,000 characters. With sign-in it also keeps a bounded memory of derived data only (interests, embeddings, timestamps), never your words, capped at 500 conversations per person. Related past conversations come back by embedding similarity, discounted by age, and the full design is in the README.

Alexa+, HTTPS MCP and deployment

Knots is built to be called by Alexa+ as an MCP client over Streamable HTTP. The server is packaged with Docker and fronted by Caddy for automatic HTTPS. MCP servers can only respond when called, so Knots surfaces recommendations when Alexa+ asks (for instance "what are we doing today?"), not by pushing notifications.

Current state. The Alexa+ add-on is not deployed yet. I hit AWS role-assumption (IAM) problems while setting it up and did not get it working in time. What exists today:

The MCP server runs end to end. All seven tools (knots_partner, rate_pick, get_recommendations, extract_interests, search_content, what_do_you_remember, forget_me) are served over Streamable HTTP with real Bedrock, TMDB, YouTube and arXiv calls.
The Explore web UI simulates Alexa+. It plays the role of the MCP client: you type what you'd say to Alexa, and it calls the same tools and renders the same picks, notes and spoken line that Alexa+ would receive. So the full recommendation loop is exercised, but through my UI rather than a real device.
The add-on and MCP descriptions are written for integration. The add-on definition and the tool descriptions lay out how Alexa+ would discover the server, which tool it calls for "what are we doing today?", and what comes back. Connecting it is a deployment step, not a redesign.
The Alexa custom-skill endpoint is implemented but not tested against a live device.

Next step: fix the role-assumption setup, register the add-on, and test on a real Alexa+ device.*

Permissions and privacy

The user ID comes from a verified bearer token or a signature-checked Alexa request, never from the caller, and is stored only as a salted hash.
what_do_you_remember shows what's held and forget_me deletes it, including logged feedback.
Embeddings aren't anonymous and stored themes can be sensitive, so the store is treated as personal data.

A note on trust

Why should you believe a recommendation? Because the note is honest, not just persuasive.

The note is where Knots makes its case, but it isn't allowed to make things up. A model writes it from nothing except the catalog's own facts and your own words. Any note with a number, name or quotation that doesn't trace back to that data is rejected, and Knots falls back to a plainer one. Behind every card sits an evidence ledger you can ask about ("why did you pick that?"), and a card with no evidence is never shown. When a paper is a preprint, Knots says so. When nothing clears the bar, it says so and asks what you want instead of guessing. A blank is honest, and a confident lie is not.

Product spec

The joint set: entertaining ∩ educative

Knots recommends from the overlap of two circles: what's entertaining and what's educative. A pick has to belong to both, or be a deliberate bridge between them. A drama that makes recursion felt, a documentary that grounds it, and a paper or explainer that formalizes it all live in the joint set. Knots turns down pure entertainment with no tie to your research, and pure homework that never earns its place on a screen. The watch-and-learn pair is this idea in its smallest unit.

Room for side quests

A recommender locked to your topic gets repetitive, so side quests are part of the product, and they are not random:

Gem slot: a lesser-known (3,000 votes or fewer), highly rated (7.0+) title.
Blockbuster slot: a widely seen title that must still be at least 60% as relevant as the best match for its topic.
Challenge me (opt-in): one slot for a serious view that complicates yours, with a note on what it pushes back on.

Each reserved slot appears only if a qualifying candidate exists, all of them sit in the first three cards, and none can skip the quality floor.

Structure vs spontaneity

Structure comes from your concepts, the receipts and the floor. Spontaneity comes only from the reserved slots.

Entertaining to educative scale

The best format is chosen per interest, so human and narrative themes lean to film, abstract mechanisms lean to explainers, and "what do the studies say" leans to papers. All three stay possible, and results are grouped as watch / read / explore.

Controls

Opt-in "challenge me", an explicit "not for me" verdict on any pick, and full memory inspection and deletion.

Friction log

Cold start killed the famous algorithms and forced a content-based redesign.
The old tag-overlap score was worse than chance (AUC 0.32), so I replaced it with a ranker fitted on my own labels.
LLM "unstated themes" were only 44% supported when I first labelled them, which led to the theme judge and to withholding themes it can't trust.
A "creativity" metric rewarded far-fetched picks, so I removed it and made grounding the headline.
Over-long input silently dropped the newest text, a bug now fixed: it keeps the latest.
Accuracy against random barely separated rankers, because retrieval and the floor already filter candidates, so I moved the argument to grounding and overreach.
Papers are the weakest format (28% of shown papers judged good, against 63% for documentaries).
The best signal about what a film is about lives in communities (Reddit, Letterboxd) that restrict access, so there's no Reddit source yet.
"Proactive" is hard in MCP, since a server only responds when called.
Labels came from synthetic sessions and one labeller. They improve general judgement, but they don't capture real users' phrasing or taste.
LLM-assisted notes can circle back into the prompt through the few-shot guidance, so I kept sessions 26 to 30 hand-written.
Receipts vs the note. Showing evidence on every card made the answer read like a citation list, so evidence became an internal ledger and the note does the persuading.
A strict floor can leave nothing to say. I haven't yet measured how often that happens, and it's the next thing I'm measuring.
YouTube API quota ran out during live testing. The explainer source stopped returning results once the daily quota was used up, so picks that depend on YouTube went missing. The solution: cached results to avid repeating requests.
Alexa+ add-on blocked by AWS role-assumption (IAM) errors. I couldn't get the add-on registered, so I built the Explore web UI to act as the MCP client and kept the add-on and tool descriptions ready for integration. The server itself ran fine; the blocker was the permissions setup around the Alexa side

What we learnt

Content-based methods are the honest answer when you have one user.
Labels beat hand-set rules: a six-weight model fitted on 24 sessions beat my priors.
The extractor mattered more than any ranker tweak.
Accuracy hides what matters. Grounding and overreach separated systems that accuracy couldn't.
Trust is a design requirement, not a feature.
One labeller and 24 sessions is a real limit, so every number here carries an interval and a caveat.
Media and research aren't opposites. Twenty seconds of cinema taught me what a shelf of books couldn't.

What's next

A feed: Knots surfaces recommendations continuously, not only when asked.
An Alexa question: after the picks, Knots asks a follow-up so the loop keeps tightening.
Label the pairs and train the pair scorer, then label the holdout themes for an honest judge number.
Evaluate the rolling window and memory on multi-conversation sessions.
Wire up the remaining online signals (engage, reformulate) and add a real exploration slot so the online gap is measurable.
Reddit and X activity as continuous signals, then collaborative filtering once there are enough users to build a utility matrix.

Built with

Python, MCP, Alexa+, Amazon Bedrock, Cohere Embed v4, TMDB API, YouTube Data API, arXiv API, OpenAlex, SQLite, Docker, Caddy

Built With

alexa+
amazon-bedrock
hhtpx
mcp
python
tmdb-api
youtube-data-api

Knots