HAVEN

Haven reconstructs explainable working context so AI resumes your projects instead of restarting every conversation.

Comment

Inspiration

HAVEN is an explainable, local-first AI memory and context engine that reconstructs Working Context rather than simply retrieving memories. It combines AI-driven knowledge extraction with deterministic retrieval, ontology reasoning, canonical memory management, and transparent ranking to help Large Language Models resume long-running projects with reliable, inspectable context.

The Problem

Large Language Models are excellent at reasoning within a conversation, but they struggle to resume long-term work.

When returning to a project after days or weeks, we repeatedly found ourselves explaining:

  • Previous architectural decisions
  • Project goals
  • Implementation progress
  • Unresolved design questions
  • Important constraints
  • Why earlier decisions had already been made

Existing AI memory systems partially solve this problem by retrieving semantically similar memories. However, during development we observed several limitations:

  • They retrieve individual memories, not the complete working context.
  • Retrieval decisions are usually driven by embedding similarity and behave like a black box.
  • Users cannot understand why memories were selected or rejected.
  • Duplicate and conflicting memories accumulate over time.
  • Retrieved memories often lack the surrounding information required to continue complex work.

Key Observation

We realized the real problem was not remembering facts.

The real problem was reconstructing enough context for an AI to continue reasoning exactly where it left off.

A memory such as:

"Use PostgreSQL for billing."

is rarely useful by itself.

The AI also needs to understand:

  • why that decision was made,
  • what alternatives were considered,
  • what constraints influenced the decision,
  • what tasks remain,
  • which questions are still unresolved, and
  • how that decision fits into the current state of the project.

Our Solution

This observation became HAVEN.

Instead of retrieving isolated memories, HAVEN reconstructs a complete Working Context before an AI begins reasoning.

The system combines:

  • AI-powered knowledge extraction
  • Deterministic retrieval
  • Ontology reasoning
  • Canonical memory management
  • Explainable ranking
  • Structured context assembly

Every retrieval decision is traceable.

Every ranking factor is inspectable.

Every rejected memory includes a recorded reason.

Rather than asking users to trust a black-box retrieval system, HAVEN makes every stage of memory retrieval transparent.

Our goal was never simply to help AI remember.

Our goal was to help AI resume work.

What it does

Overview

HAVEN is a local-first AI memory and working-context engine that allows Large Language Models to continue long-running projects without repeatedly re-explaining previous conversations.

Unlike traditional memory systems that retrieve isolated memories, HAVEN reconstructs a complete Working Context describing the current state of a project before any reasoning begins.

This allows AI assistants to resume ongoing work with significantly richer context than conversation history or semantic memory retrieval alone.

Knowledge Ingestion Pipeline

Every conversation passes through a three-stage semantic pipeline.

1. Extractor

  • Identifies atomic facts from conversations.
  • Preserves provenance for every extracted memory.

2. Classifier

  • Assigns one of 18 structured memory types.
  • Identifies canonical concepts.
  • Produces structured metadata for retrieval.

3. Importance Scorer

  • Estimates the long-term importance of each memory.
  • Helps prioritize future retrieval.

These are the only AI-powered semantic stages.

After semantic understanding is complete, the remainder of the pipeline is deterministic.

Canonical Memory Management

HAVEN continuously maintains a clean and evolving knowledge base instead of endlessly accumulating memories.

Every extracted memory is compared against existing knowledge and classified as:

  • NEW
  • CONFIRM
  • UPDATE
  • SUPERSEDE

Repeated confirmations strengthen existing knowledge rather than creating duplicates.

This allows the knowledge base to evolve while remaining compact and internally consistent.

All memories are stored locally as Markdown files with YAML frontmatter, making the system:

  • Transparent
  • Portable
  • Human-readable
  • Version controllable
  • Compatible with Obsidian

Deterministic Retrieval Pipeline

When a user asks a question, HAVEN performs deterministic retrieval rather than relying solely on embedding similarity.

The retrieval pipeline performs:

  1. Concept detection
  2. Alias resolution
  3. Ontology traversal
  4. Graph activation spreading
  5. Deterministic keyword matching
  6. Multi-factor ranking
  7. Confidence-based acceptance

Unlike many retrieval systems, HAVEN can intentionally return no memories when confidence is insufficient instead of forcing an answer.

Working Context Reconstruction

Retrieved memories are not injected directly into the prompt.

Instead, they are assembled into a structured Working Context containing:

  • Active goals
  • Previous decisions
  • Current tasks
  • Open questions
  • Supporting references
  • Current project state

This provides significantly richer context than isolated memory retrieval and allows the AI to continue ongoing work rather than simply recalling previous facts.

Browser Integration

HAVEN integrates directly into ChatGPT through a Chromium browser extension.

The extension provides two primary actions:

Use Haven

Retrieves the reconstructed Working Context and injects it into the conversation before the AI begins reasoning.

Remember

Extracts newly learned information after a conversation finishes and updates the knowledge base.

HAVEN also includes a Query Rewrite Assistant that selectively suggests improved continuation queries whenever a clearer formulation would improve retrieval quality.

Explainability

Every major stage of the system is inspectable.

HAVEN includes:

  • Retrieval Inspector
  • Write Inspector
  • Benchmark Explorer
  • Interactive Dashboard

These tools expose the same internal data structures used by the retrieval pipeline itself.

Users can inspect:

  • Why a memory matched
  • Why it ranked where it did
  • Why other memories were rejected
  • How canonical memories changed
  • How each write operation modified the knowledge base

Explainability is not an additional feature—it is part of the system architecture.

How we built it

Design Philosophy

HAVEN is built around one central engineering principle.

Use AI where semantic understanding is required. Use deterministic algorithms everywhere else.

Large Language Models excel at understanding language.

Traditional software excels at:

  • consistency,
  • reproducibility,
  • explainability,
  • debugging,
  • maintainability.

HAVEN combines both.

Architecture Overview

Layer Purpose
Knowledge Ingestion Converts conversations into structured knowledge
Canonical Memory Management Prevents duplicates and maintains evolving knowledge
Ontology Retrieval Retrieves relevant knowledge using deterministic algorithms
Working Context Builder Reconstructs project state before reasoning
Browser Extension Integrates HAVEN directly into ChatGPT
Dashboard & Inspectors Explains every retrieval and write decision

The overall architecture is:

Conversation
        │
        ▼
Knowledge Ingestion
        │
        ▼
Canonical Knowledge Base
        │
        ▼
Ontology + Retrieval
        │
        ▼
Working Context
        │
        ▼
Large Language Model

Rather than relying on one large prompt, every responsibility is isolated into independent pipeline stages.

1. Knowledge Ingestion

The write pipeline consists of three specialized AI stages.

Extractor

Extracts atomic facts while preserving provenance.

Classifier

Assigns:

  • memory type,
  • canonical concept,
  • structured metadata.

Importance Scorer

Estimates long-term retrieval value.

Separating these responsibilities made every stage:

  • independently benchmarkable,
  • easier to debug,
  • easier to replace,
  • more reliable than a single monolithic prompt.

After these three stages, every remaining component is deterministic.

2. Canonical Memory Management

A long-term memory system cannot simply accumulate memories forever.

HAVEN introduces a Canonical Matcher that determines whether new information represents:

  • New knowledge
  • Confirmation
  • Update
  • Superseded knowledge

Instead of storing every extracted statement independently, the knowledge base continuously consolidates information into cleaner canonical memories.

This reduces duplication while preserving historical context.

3. Ontology-Based Retrieval

Traditional vector databases treat memories as isolated documents.

HAVEN instead builds an ontology describing relationships between concepts.

During retrieval the system performs:

  1. Concept detection
  2. Alias resolution
  3. Ontology traversal
  4. Activation spreading
  5. Keyword retrieval
  6. Deterministic ranking
  7. Confidence filtering

Multiple independent scoring factors contribute to the final ranking instead of a single similarity score.

Most importantly, every retrieval decision remains completely inspectable.

4. Working Context Assembly

Retrieval alone does not provide enough context for long-running projects.

Accepted memories are assembled into a structured Working Context containing:

  • Goals
  • Decisions
  • Tasks
  • Open Questions
  • References
  • Current Project State

This Working Context becomes the context injected into the AI before reasoning begins.

Instead of asking the language model to infer project state from scattered memories, HAVEN reconstructs it explicitly.

5. Browser Extension

To make HAVEN usable during everyday AI conversations, we built a Chromium browser extension that integrates directly into ChatGPT.

The extension allows users to:

  • Retrieve Working Context
  • Inject context into conversations
  • Remember new knowledge
  • Inspect retrieval results
  • Configure remote servers
  • Receive optional Query Rewrite suggestions

The extension communicates with a FastAPI backend while supporting both local and remote deployments.

6. Benchmarking and Evaluation

Rather than relying on curated demonstrations, we built a benchmark suite evaluating retrieval performance across hundreds of categorized retrieval tasks.

The benchmark compares multiple retrieval strategies, including:

  • HAVEN Full
  • Retrieval-only ablations
  • Dense embedding retrieval
  • BM25
  • Recency retrieval

During development we also audited the benchmark itself to ensure that every reported result accurately reflected real system behavior.

Improving the benchmark proved almost as valuable as improving the retrieval pipeline itself.

Technology Stack

Component Technology
Backend FastAPI (Python)
Frontend HTML, CSS, JavaScript
Browser Extension Chromium Manifest V3
AI Models Qwen via DashScope
Knowledge Storage Markdown + YAML
Documentation GitHub Pages
Deployment Alibaba Cloud ECS + nginx + systemd

The result is a complete end-to-end platform capable of ingesting conversations, maintaining structured long-term knowledge, reconstructing working context, and integrating directly into existing AI workflows.

Challenges we ran into

1. Preventing Long-Term Memory Degradation

One of the hardest engineering problems was preventing the knowledge base from slowly degrading over time.

A naïve memory system stores every extracted fact independently.

As conversations grow, this naturally creates:

  • Duplicate memories
  • Conflicting information
  • Outdated knowledge
  • Increasing retrieval noise
  • Lower retrieval precision

Simply storing more memories eventually makes retrieval worse.

To solve this, we designed a Canonical Memory Management layer that determines whether newly extracted information should:

  • Create new knowledge (NEW)
  • Confirm existing knowledge (CONFIRM)
  • Update previous knowledge (UPDATE)
  • Replace outdated knowledge (SUPERSEDE)

Instead of endlessly accumulating memories, HAVEN continuously refines its knowledge base.

Designing this system while keeping retrieval deterministic and explainable required extensive experimentation.

2. Choosing Where AI Should Stop

Another major challenge was deciding how much of the system should rely on language models.

Early experiments delegated larger portions of the pipeline to LLMs.

While this increased flexibility, it also introduced:

  • inconsistent outputs,
  • prompt sensitivity,
  • higher costs,
  • difficult debugging,
  • limited explainability.

We eventually adopted a much stricter architectural principle.

Use AI only where semantic understanding is required. Use deterministic algorithms everywhere else.

This decision shaped nearly every subsystem inside HAVEN.

Although it required significantly more engineering effort, it made the platform reproducible, inspectable, and significantly easier to improve over time.

3. Building Working Context Instead of Memory Retrieval

Perhaps the biggest architectural challenge was realizing that memory retrieval alone does not solve long-running collaboration.

A retrieved memory such as:

"Use PostgreSQL."

is rarely enough.

The AI also needs to understand:

  • why that decision was made,
  • which alternatives were rejected,
  • what goals it supports,
  • what constraints exist,
  • what work remains,
  • and how it relates to the current project.

This shifted our focus from building a retrieval engine to building a Working Context reconstruction engine.

Instead of optimizing only retrieval quality, HAVEN reconstructs the current state of a project before any reasoning begins.

This ultimately became the defining architectural idea behind the project.

4. Making Retrieval Explainable

Most AI memory systems expose very little information about how retrieval actually works.

We wanted every retrieval decision to be inspectable.

This required designing an entirely separate explainability layer exposing:

  • Retrieval Inspector
  • Write Inspector
  • Benchmark Explorer
  • Retrieval traces
  • Write traces
  • Ranking breakdowns
  • Rejection reasons

These tools significantly increased development effort but became invaluable during debugging.

Every retrieval improvement we made was driven by understanding why retrieval behaved the way it did rather than treating the system as a black box.

5. Trusting the Benchmark Before the Results

Building a benchmark turned out to be almost as challenging as building the retrieval system itself.

During development we discovered issues within our own evaluation framework that could have produced misleading results.

Rather than ignoring those problems, we treated benchmark correctness as part of the project.

Improving the benchmark ultimately improved the retrieval system itself.

One of our biggest engineering lessons was that evaluation infrastructure deserves the same level of scrutiny as production code.

Accomplishments that we're proud of

Reconstructing Working Context Instead of Retrieving Memories

The accomplishment we are most proud of is introducing Working Context as the primary retrieval target.

Instead of returning isolated memories, HAVEN reconstructs:

  • goals,
  • decisions,
  • tasks,
  • open questions,
  • project state,
  • supporting references,

allowing AI systems to resume long-running work with significantly richer context.

We believe this is a fundamentally different approach to AI memory.

Deterministic and Explainable Retrieval

Nearly the entire retrieval pipeline is deterministic.

Every retrieval decision can be inspected, including:

  • Why a memory matched
  • Why it ranked where it did
  • Why another memory was rejected
  • How canonical memories changed
  • How new knowledge evolved

Rather than asking users to trust retrieval, HAVEN allows them to verify it.

Building a Complete Platform

HAVEN grew far beyond a retrieval algorithm.

We built an end-to-end platform consisting of:

  • AI knowledge ingestion pipeline
  • Canonical memory management
  • Ontology-based retrieval
  • Working Context reconstruction
  • Chromium browser extension
  • FastAPI backend
  • Interactive dashboard
  • Explainability tools
  • Benchmark framework
  • Cloud deployment
  • Documentation website

Every component works together as a complete AI memory ecosystem.

Local-First Knowledge Ownership

All knowledge is stored as Markdown with structured YAML metadata.

This makes the knowledge base:

  • Human-readable
  • Portable
  • Editable
  • Version controllable
  • Compatible with Obsidian
  • Independent of proprietary formats

Users always retain ownership of their knowledge.

Benchmark-Driven Engineering

Rather than relying on carefully selected demonstrations, we continuously evaluated HAVEN using a benchmark suite covering hundreds of categorized retrieval tasks.

Architectural decisions were guided by measurable improvements instead of intuition alone.

This benchmark-first approach significantly improved both retrieval quality and engineering confidence.

Project Scale

During the hackathon we built:

  • AI-powered knowledge ingestion pipeline
  • 18 structured memory types
  • Canonical memory management
  • Ontology graph
  • Deterministic retrieval engine
  • Working Context reconstruction
  • Chromium browser extension
  • FastAPI backend
  • Explainability dashboard
  • Benchmark framework
  • Cloud deployment
  • Documentation platform

Together these components form a complete context reconstruction system rather than a standalone retrieval model.

What we learned

AI Systems Need Strong Software Engineering

One of our biggest lessons was that building reliable AI systems is often more about software engineering than prompt engineering.

Language models provide excellent semantic understanding.

Deterministic software provides:

  • consistency,
  • reproducibility,
  • maintainability,
  • explainability,
  • reliability.

The strongest parts of HAVEN emerged from combining both rather than relying exclusively on either approach.

Context Is More Valuable Than Memory

Perhaps the biggest insight from the project was that users rarely need a remembered fact.

They need enough surrounding context to continue reasoning without reconstructing previous conversations themselves.

This realization fundamentally changed our architecture.

Every major subsystem eventually evolved toward reconstructing Working Context rather than retrieving isolated memories.

Explainability Accelerates Development

Every retrieval trace we exposed became another debugging tool.

Instead of guessing why retrieval failed, we could inspect:

  • ranking factors,
  • activation scores,
  • canonical updates,
  • rejection reasons,
  • retrieval decisions.

Explainability benefited both users and developers.

It transformed retrieval optimization from trial-and-error into an engineering process.

Benchmarks Must Be Trusted

Building evaluation infrastructure taught us that benchmark correctness is just as important as benchmark scores.

A flawed benchmark creates false confidence and can easily send development in the wrong direction.

Treating the benchmark as production software ultimately produced a much stronger retrieval system.

Long-Term AI Memory Is a Knowledge Management Problem

Initially we thought we were building a retrieval engine.

By the end of the project we realized we were actually building a knowledge management system.

Retrieval is only one component.

Maintaining consistent knowledge, reconstructing project state, evolving memories over time, and explaining retrieval decisions proved equally important.

This insight completely changed how we think about AI memory systems.

What's next for HAVEN

Smarter Knowledge Evolution

Our next major milestone is improving how HAVEN handles changing knowledge.

Future versions will expand semantic contradiction detection and automatically evolve canonical memories as projects change over time while preserving explainability.

Richer Working Context

The current Working Context already reconstructs project state from retrieved memories.

Future versions will incorporate:

  • freshness detection,
  • richer project reasoning,
  • incremental continuation workflows,
  • improved context reconstruction,

while preserving deterministic behavior.

More Integrations

We want HAVEN to become independent of any single AI platform.

Future integrations include:

  • Claude
  • Gemini
  • IDE workflows
  • Additional browsers
  • More AI providers

allowing one persistent knowledge base to power multiple assistants.

Larger Evaluation Framework

We plan to continue expanding our benchmark suite with additional datasets covering:

  • Software engineering
  • Research workflows
  • Personal knowledge management
  • Long-running conversations
  • Multi-session collaboration

Our goal is to make the benchmark itself a useful resource for evaluating future AI memory systems.

Long-Term Vision

Ultimately, we envision HAVEN becoming the persistent context layer for AI.

Language models will continue to improve and change, but users should never lose their knowledge, project state, or working context because they switched providers, started a new conversation, or returned weeks later.

Rather than acting as another memory database, HAVEN aims to become the infrastructure layer that enables AI assistants to maintain persistent knowledge, reconstruct working context, explain every retrieval decision, and collaborate continuously across conversations, tools, and even different AI models.

Built With

Share this project:

Updates

Submission history