Inspiration
Most of us have hundreds of saved TikToks - restaurants in Shibuya, wardrobe ideas, workout splits we never started - each one a small, half-formed goal. Agent platforms don't help here: they track Runs, sessions, and files, but nothing understands which fragmented tasks belong to the same real-world goal. Ask an agent something, then a follow-up, then something unrelated, and it either treats each message like a stranger or dumps everything into one undifferentiated memory. We realized this isn't a model problem: it's an infrastructure problem.
What it does
GoalThread sits between an agent platform's Run history and its memory layer. For every new Run, it decides whether to:
- Merge into an existing Goal Thread (e.g., a Shibuya question merges into an active Tokyo Trip thread)
- Stay separate as a new thread, if unrelated (a workout request shouldn't contaminate a travel thread)
- Fork when the goal itself has shifted ("actually, forget Tokyo, let's do Seoul" forks a new thread and closes the old one)
How we built it
We read the spec and explored the existing codebase first — how Agents, Runs, and messages are stored, and the exact moment a task completes. From there we added two new concepts: a Goal Thread (a group of related Runs) and a Decision (why each Run landed where it did). The core is a two-tier decision engine. Tier 1 checks four deterministic signals — shared keywords, referential phrases like "those" or "continue," shared files, and same-Agent continuity. If 2+ signals agree, it merges automatically with no AI call. If only one signal fires, Tier 2 asks the model one focused question: "does this belong with any existing thread?" If none fire, it's a new thread. An explicit goal change ("forget X, I want Y") triggers a fork and closes the old thread.
We enforced strict isolation - a thread can only ever see its own messages, never a parent's or sibling's - and wrote a test specifically to try to break that. We wired the engine in right after a Run completes, wrapped so a failed AI call never blocks task completion, and built 4 API endpoints plus a "Goal Threads" sidebar panel to make grouping visible. We tested with unit tests and a 41-example dataset (Tokyo trip, gym routine, book recs, deliberately tricky cases), hitting 98% accuracy. Along the way we fixed thread over-splitting in plain chat, broken title generation, and a few Windows-only environment issues.
Challenges we ran into
Plain back-and-forth chat initially split into a new thread almost every message; adding the same-Agent-continuity signal mostly fixed it, but forks sometimes still leaked the old goal's name into the new thread's memory. We hit one real, unresolved trade-off: an older, quiet thread occasionally got overlooked in favor of a recent one — widening the candidate pool fixed that case but caused more wrong merges elsewhere, so we reverted it and documented the limitation instead of hiding it. We also caught our own test bugs (a false-alarm label mismatch, a test that hid failures by aborting early), plus Windows-specific issues: a .env file that wasn't loading and a Codex CLI launch path we had to fix carefully to avoid a security shortcut. We also spent real time hunting down stray duplicate servers hogging ports.
Accomplishments that we're proud of
- 98% accuracy on a deliberately adversarial test set, not just easy cases
- A leak-proof thread boundary, proven with a test designed to break it
- Getting FORK right: detecting genuine goal shifts, not just relatedness
- Documenting a real limitation honestly instead of shipping a false fix
- Graceful degradation: a failed AI call never breaks the underlying task
What we learned
We started thinking about this as a "memory" feature and ended up thinking about it as a goal-aware platform: treating user intent as a first-class object Runs attach to, not something inferred fresh each message. The most convincing part wasn't the merging logic itself but the evidence trail behind each decision, which turned an invisible design choice into something a user could trust. The core lesson: memory in agent systems is a design problem, not a storage problem. Storing everything is easy; understanding what belongs together is not — and the fix is a clear, testable decision at the right seam, not a bigger system.
What's next for GoalThread
- A better fix for the recency/overlooked-thread trade-off we documented
- Letting users manually correct merges/forks, feeding that back into scoring
- Modeling thread staleness and resumption, not just creation and forking
- Growing the test set beyond our 41 curated examples with real usage
- A real ingestion path (e.g. TikTok export) beyond our mock dataset
Built With
- byteplus
- deepseek
- docker
- fastify
- json
- modelark
- node.js
- openai-codex
- react
- typescript
- vite
- vitest
- zod
Log in or sign up for Devpost to join the conversation.