Latch — Prospective Memory for Agents

Typed prospective memory that notices the right future cue and acts exactly once

Comment

Latch: Prospective Memory for AI Agents

## Inspiration

Most AI memory systems focus on remembering the past. They store conversations, retrieve facts, and recall user preferences when someone asks a related question.

However, real assistants also need to remember the future.

A user might say:

“When legal approves version 7 of the contract, prepare the redline for Dana—but wait for finance approval too, and ask me before creating the draft.”

This is not simply information to retrieve later. It is a future commitment with conditions, constraints, updates, safety policies, and a specific action.

I built Latch to explore a different kind of agent memory: prospective memory. This is the ability to remember what must happen, recognize when the correct future condition arrives, ignore misleading events, and act safely without requiring the user to repeat the instruction.

The central question behind the project was:

Can an AI agent reliably remember an intention across sessions, detect the correct future cue, explain why it acted or remained silent, and execute the action exactly once?

———

## What It Does

Latch converts ordinary conversational commitments into structured, versioned Intention Programs.

Each Intention Program can contain:

  • The action the user wants performed
  • Exact event or state-based triggers
  • Multiple conditions that must all become true
  • Conditions that block or invalidate the action
  • Time-based and absence-based rules
  • Human approval requirements
  • User interruption preferences
  • Provenance linking decisions to the user’s original words
  • A forgetting policy for removing completed intentions from active memory

Instead of continuously placing an entire conversation history into the model context, Latch keeps a small set of relevant active intentions and retrieves only those associated with an incoming event.

The application includes:

  • A conversational intention compiler
  • Immutable intention-version history
  • Active-memory inspection
  • A ten-step deterministic simulator
  • A human-approval interface
  • An event and decision timeline
  • Duplicate-event suppression
  • Exactly-once action execution
  • Absence-trigger detection
  • Purpose-based forgetting
  • A reproducible 60-scenario evaluation
  • A cloud-service and model-routing proof screen

———

## The Demonstration Scenario

The main demonstration follows a contract-approval workflow.

The user initially asks the agent to prepare a DPA redline when legal approves the document. During a later session, the user revises the instruction and adds finance approval as another required condition.

Latch then processes several events:

  1. A marketing approval containing similar wording is rejected.
  2. A legal response for an outdated document version is rejected.
  3. Legal approval for the correct contract and document version is accepted.
  4. Finance approval for an unrelated deal is rejected.
  5. The agent remembers that the user is inside a focus block.
  6. Finance approval for the correct deal completes the compound trigger.
  7. The notification is deferred until the focus block ends.
  8. The external draft is presented for human approval.
  9. The approved action is executed exactly once.
  10. A duplicate webhook and repeated approval cannot create another action.
  11. A separate 48-hour absence rule produces an internal escalation draft.

After completion, the intention leaves active memory while its immutable audit history remains available.

———

## How I Built It

### Typed Intention Programs

The core of Latch is a structured representation of future commitments.

An Intention Program records:

  • Source message and quoted evidence
  • Version number
  • Current lifecycle state
  • Allowlisted action and arguments
  • Trigger predicates
  • Entity constraints
  • Inhibitors
  • Absence rules
  • Approval policy
  • Interruption preferences
  • Forgetting rules
  • Model and prompt metadata

Updates never silently overwrite an existing intention. Instead, Latch creates a complete new version and marks the previous version as superseded. This preserves the exact instruction that authorized every future decision.

### Deterministic Runtime

I deliberately kept critical decisions outside the language model.

Exact identifiers, document versions, lifecycle transitions, approval requirements, duplicate detection, action claiming, and forgetting behavior are evaluated deterministically.

Qwen is used where language understanding is valuable:

  • Compiling natural-language commitments
  • Producing structured intention data
  • Creating concise explanations
  • Generating semantic embeddings
  • Supporting bounded semantic comparison

The model can propose an interpretation, but deterministic validation decides whether that interpretation is safe to activate.

### Qwen Integration

All model access is isolated behind a Qwen adapter.

The full reference backend supports Qwen Cloud for structured compilation and bounded semantic adjudication. The public zero-cost deployment uses Qwen3 for intention compilation and embeddings while preserving the same typed program contract.

This separation allows the runtime to switch model providers without changing the safety-critical state machine.

### Reference Backend

The full reference implementation uses:

  • FastAPI
  • Python and Pydantic
  • PostgreSQL
  • pgvector
  • SQLAlchemy and Alembic
  • A background scheduler
  • Qwen Cloud integration
  • Alibaba Cloud OSS integration

PostgreSQL stores intentions, immutable versions, events, decisions, approvals, actions, evaluation runs, and replay evidence.

### Public Cloudflare Deployment

The public demonstration is deployed using:

  • Cloudflare Pages
  • Cloudflare Workers
  • D1
  • Workers KV
  • Durable Objects
  • R2
  • Queues
  • Vectorize
  • Workers AI
  • AI Gateway
  • Workflows
  • Cron Triggers

Durable Objects coordinate atomic usage limits and exactly-once claims. D1 stores the application state and audit records. Queues process events asynchronously. Vectorize stores semantic cue embeddings. Workflows generate replay snapshots that are preserved in a private R2 bucket.

The public demo also includes strict daily limits so it can remain accessible without enabling a paid cloud plan.

———

## Challenges I Faced

### Balancing Recall and Restraint

The hardest problem was not remembering more—it was deciding when not to act.

A semantic memory system may retrieve anything that sounds related. That behavior is dangerous for future actions because a message containing the word “approval” does not necessarily satisfy a particular contract-approval condition.

I addressed this by separating:

  • Event channel
  • Event type
  • Entity identifiers
  • Document version
  • Lifecycle state
  • Inhibitors
  • Optional semantic similarity

Semantic relevance can help rank a candidate, but it cannot override a failed hard constraint.

### Handling Cross-Session Updates

Users frequently revise instructions later:

“Actually, wait for finance approval too.”

Mutating the original memory would make it difficult to determine which instruction authorized an action. Latch therefore creates immutable versions and evaluates only the active version.

This made revision, cancellation, and supersession visible and auditable.

### Preventing Duplicate Actions

Real event systems retry messages. Users may double-click approval buttons, and workers may restart after partially completing an operation.

A simple “check whether the action exists” implementation is not enough because two requests may perform that check simultaneously.

Latch combines:

  • Stable event identifiers
  • Completed-step tracking
  • Idempotency keys
  • Atomic action claims
  • Durable Object coordination
  • Persistent action records

This ensures that replaying an event or approval does not create a second action.

### Remembering Missing Events

Some intentions become relevant because something did not happen.

For example:

“If legal does not respond within 48 hours, prepare an escalation to Mark.”

There is no incoming event representing that absence. The runtime must persist a deadline, wake at the correct time, verify that the expected response is still missing, and create the escalation only once.

This required treating absence as a first-class trigger rather than a special reminder.

### Explaining Silence

An agent must explain why it rejected an event just as clearly as why it accepted one.

Latch records evidence for both outcomes, including:

  • Wrong channel
  • Wrong entity
  • Stale document version
  • Failed status predicate
  • Active inhibitor
  • Focus-time deferral
  • Duplicate-event rejection

This became one of the project’s most important features because reliable agents must make non-action observable.

### Maintaining a Zero-Cost Public Demo

The deployment needed to remain publicly testable without creating unexpected cloud charges.

I added application-level limits for:

  • Daily mutations
  • Qwen requests
  • Workflow executions
  • Replay creation
  • Persistent writes

The deterministic scenario continues to work even if the model quota is exhausted.

———

## What I Learned

### Agent Memory Is a Control System

Memory for an autonomous agent is not only a retrieval problem.

Reliable future behavior requires:

  • State
  • Event routing
  • Time
  • Policy
  • Versioning
  • Invalidation
  • Approval
  • Idempotency
  • Forgetting

A vector database can help find candidates, but it should not be the component that decides whether an external action is authorized.

### Forgetting Is Part of Correctness

Keeping every memory active forever increases context size and creates opportunities for outdated instructions to influence future decisions.

Latch separates:

  • Active hot memory
  • Completed cold memory
  • Immutable audit evidence

When an intention reaches a terminal state, it leaves the active retrieval path but remains available for debugging and replay.

### Models Should Interpret, Not Authorize

Qwen is excellent at translating natural language into structured meaning. However, language-model output should not directly authorize external writes.

The safest architecture was:

  1. Let Qwen propose a structured interpretation.
  2. Validate it against a strict schema.
  3. Evaluate exact conditions deterministically.
  4. Require human approval for external actions.
  5. Claim and execute the action exactly once.

### Non-Action Is a Product Feature

Rejecting an incorrect event is not a failure. Deferring an interruption during focus time is not inactivity. Forgetting a completed intention is not data loss.

These are intentional decisions that should be visible in the interface and evaluation results.

### Evaluation Must Include Distractors

A memory system can appear successful if it is tested only with positive examples.

I created the PM-Mini v1 dataset with 60 deterministic scenarios covering:

  • Exact cues
  • Similar but incorrect entities
  • Stale document versions
  • Inhibitors
  • Absence rules
  • Cancellation
  • Duplicate delivery

On this published dataset, Latch achieved:

  • Prospective-memory F1: 1.000
  • False alarms: 0
  • Missed cues: 0
  • Duplicate actions: 0

These results describe the included synthetic benchmark rather than claiming general performance in every production environment.

———

## Accomplishments

I am particularly proud that Latch demonstrates:

  • Cross-session intention revision
  • Typed and versioned prospective memory
  • Exact lure rejection
  • Compound future triggers
  • Remembered interruption preferences
  • Human approval bound to a specific action
  • Exactly-once execution
  • Duplicate webhook suppression
  • Absence detection
  • Purpose-based forgetting
  • Visible explanations for action and silence
  • Reproducible baseline evaluation
  • A public, interactive cloud deployment

———

## What’s Next

Future work includes:

  • Gmail and Outlook connectors
  • Slack and Microsoft Teams events
  • Calendar and contract-system integrations
  • Recurring intentions
  • Team-based memory with scoped access
  • Richer stale-memory propagation
  • User-facing memory export and deletion
  • Additional real-world prospective-memory datasets
  • SDKs for integrating Latch with other agent frameworks

The long-term goal is to make prospective memory a reusable infrastructure layer for agents—not another chatbot feature.

———

## Closing

Most agent memory systems answer:

“What happened before?”

Latch is designed to answer a different question:

“What must happen next, what would make it relevant, and when is the agent allowed to act?”

That difference turns memory from passive retrieval into reliable, observable future behavior.

Built With

Share this project:

Updates

Submission history