Reflex

Architecture Diagram

Inspiration

Browser and computer-use agents think from scratch on every click. One LLM round trip per step, 1–3 seconds each, forever. The 100th time an agent books a meeting it's exactly as slow and expensive as the first. Humans don't work like that: repeated procedures become muscle memory, and we only think when something unexpected happens. We wanted to give agents muscle memory.

What it does

Before every step, a Reflex agent asks Moss: "have I been in exactly this situation before?" On a confident, verified match it replays the remembered action without calling the LLM. Anything unfamiliar goes to the LLM for just that one step, and the result is learned instantly.

Measured on our demo booking flow:

Run	Condition	LLM calls	Steps
1	cold	11	🧠 × 10
2	different person / day / time	1 (parse only)	⚡ × 10
3	rephrased instruction	1	⚡ × 10
4	shuffled layout	1	⚡ × 10
5	renamed labels	7	⚡⚡⚡🧠🧠🧠🧠🧠↩⚡
6	renamed again (relearned)	1	⚡ × 10

Warm runs are 9–13× faster in agent time (about 1–2 s vs 11–20 s).

How we built it

Templated state. Each page is snapshotted into interactive elements (role, label, value). Task values like name, email, day and time are replaced with slots ({name}, {day}), so a step learned for "Priya on Thursday" transfers to "Arjun on Friday".
Moss reflex index. Every learned step is a document in a Moss in-process session index: the templated state embedded with MiniLM (q8, local), with the action and expected post-state as payload. Every step queries Moss with a flow filter, then candidates are exactly rescored.
Four safety gates. A reflex only fires if all of these pass:
1. similarity ≥ 0.90;
2. exact form state match;
3. the target element exists exactly once on the live page;
4. the page lands where expected after acting.

Failing reflexes are penalised and pruned.

LLM fallback. Groq gpt-oss-120b with tool calling (click, type, select, check, done), rotating models on rate limits.
Mission-control UI. Next.js with a live step timeline (🧠 LLM / ⚡ reflex / ↩ fallback), per-step timings, a learning curve, chaos modes (shuffled layout, renamed labels), and a live "Why Moss" benchmark.

Why Moss

The lookup sits on the hot path of every agent step and replaces an LLM call, so it has to cost less than a click. Moss answers in 0.6–0.9 ms p50 across 20,016 reflexes, versus about 9.8 ms for a brute-force scan, and new reflexes are searchable about 65 ms after they're learned. No network round trip happens in the loop.

Challenges we ran into

Embeddings can't see form progress. A page with "Full name" empty vs filled embeds at cosine 0.991, so our first version happily re-typed the name forever. We added an exact form-state gate on top of Moss recall.
Moss scores are rank-fused (the top hit is always 1.0), so we use Moss for fast recall and rescore candidates with exact cosine.
Built-in Moss models returned 401 for our project, so we bring our own 384-d embeddings via session(..., "custom").
Groq free-tier token limits mid-run, solved with model rotation on 429.

Accomplishments that we're proud of

10/10 steps replayed on the second run.
Shuffled layouts: still 100% reflexes.
Renamed labels: handled safely, with rejection before acting and relearning on the next run.
18/18 end-to-end runs passing across repeated test passes.

What we learned

Retrieval speed changes what's architecturally possible. When lookup costs under a millisecond, you can put it in front of every model call and treat the LLM as the fallback instead of the default.