Inspiration
Every day, your AI agent forgets what you decided yesterday. You correct the same things, re-explain the same preferences, watch it confidently do the wrong thing in your name. Agent memory usually means "store more text". We wanted something different: an agent that builds an empirical, evidence-cited model of how its user decides — and uses it to predict your feedback before you give it.
The method grew out of our own prompt-archaeology work: classifying thousands of real human-AI interaction protocols showed that user decisions follow recurring, learnable patterns — and that the honest limit of such models (robust on recurring cases, fragile on novel ones) has to be built into the design, not footnoted.
What it does
build-your-users-mind is a local-first recipe (a method + scripts + document templates, not a framework) that lets an agent turn its own authorized interaction logs into a living preference model:
- The pipeline: extract → merge → chunk → classify → aggregate. Deterministic scripts pull only the prompts the user actually typed and redact secrets before anything is written; a manifest is bound to the exact SHA-256 of the corpus; only then does an LLM classification swarm label prompts against an 8-type taxonomy; a hard validation gate refuses to aggregate if anything is missing, malformed, or colliding.
- Feedback precognition: a 0→4 runtime loop turns the model into predictions with a confidence traffic-light — 🟢 consistent pattern: act and log it; 🟡 context-dependent: ask; 🔴 no pattern: escalate, don't guess. The system raises its hand when unsure.
- Agent-agnostic: source adapters for Claude Code, Codex CLI, Gemini/Antigravity, and Kimi feed one shared corpus — what one agent learns about the user, all agents can use.
- Measurable, not mystical:
score_predictions.pycomputes the hit-rate of predictions against real later feedback, per confidence tier; a dependency-free Cohen's-κ helper ships for inter-rater checks. - Try it in 60 seconds: a one-command synthetic demo runs the deterministic build, a pre-authored feedback-scoring loop, and the tamper gate offline — no LLM, no network, no real data.
How we built it
Stdlib-only Python, TDD (73 deterministic tests), Windows/Linux CI. Deterministic-first was the architectural bet: the model only ever sees a reduced, redacted corpus.
The part we'll keep telling people about: Codex wrote its own adapter. Commit 1e3abc4
records the delegated Codex implementation and control test on 946 real prompts. Who better to
parse an agent's log format than the agent that writes it? Codex also authored the repository's
discovery metadata; commit 0ec49df carries the public author Codex <codex@local>.
For the final Build Week hardening pass, GPT-5.6 in Codex reviewed the source adapters,
redaction boundary, corpus merging, and feedback scorer. Codex Session
019f8674-fe9a-7d91-a80f-7ee799e8ced0 found and fixed nine privacy and data-integrity defects;
the resulting 73-test suite, lint, compilation check, and offline demo all pass.
Challenges we ran into
- Four log formats, none documented. Claude is JSONL, Codex is rollout-JSONL with internal artifacts to filter, Kimi is wire-JSONL — and Gemini stores turns as Protobuf blobs inside SQLite, which meant writing a custom varint parser against an undocumented schema.
- Privacy as architecture, not promise. Fail-closed extraction, redaction before write, aggressive
.gitignorefor anything generated — a leaked API key must be structurally unable to reach the model. - Resisting our own hype. A historical inter-rater agreement of κ≈0.24 for the semantic labels is flagged in the repo, and the docs say plainly: semantic quality stays human-reviewed by design. Shipping that number was a choice.
- Silent approval is invisible. Users type corrections, not praise — so the corpus skews critical, and the model has to be calibrated for that bias.
- The demo had to be bulletproof. A cloud-sync file lock crashed re-runs of the synthetic demo on Windows; we hardened cleanup with retries and fallback run-dirs and regression-tested it, because a juror's second run matters as much as the first.
Accomplishments that we're proud of
- An OpenAI model verifiably co-authored the project — it's all in the git history.
- Predictions that cite their evidence (prompt-ID citations), with an escalation rule instead of false confidence.
- A core claim you can measure, not just believe (
score_predictions.py). - A 60-second, fully offline reproduction path for reviewers.
- Documentation in six languages, honest about which parts still lag.
What we learned
Deterministic-first beats LLM-everywhere — most of the pipeline never needed a model, and that's exactly why it's trustworthy. "Escalate, don't guess" is a safety primitive, not a UX nicety. And publishing your weakest number (κ≈0.24) earns more trust than hiding it.
What's next for build-your-users-mind
A companion harness-hook module (SessionStart reminders that surface the 0→4 loop automatically), a searchable FTS index over the redacted cross-agent corpus ("what has any of my agents already discussed with this user?"), a fresh representative inter-rater evaluation to replace the historical κ, refreshed translations, and a first stable tag once the semantic gates pass.
Built with
python · json-schema · sqlite · protobuf · github-actions · ruff · unittest · openai-codex · claude · gemini · kimi · markdown · git · windows · linux
Log in or sign up for Devpost to join the conversation.