MIRA (Memory Integrated Reasoning Architecture)

A working memory loop, not a longer prompt: MIRA carries decisions and corrections across sessions, retires what's outdated, and shows what shaped each answer.

Comment

Inspiration

For a long time, I had been searching for a realistic way to contribute to AI.

I considered research areas such as model pretraining, distillation, and architecture design. However, I kept returning to the same economic problem: how does an individual with limited funding compete in areas where progress can require thousands of GPUs and billions of dollars? I was unlikely to build a foundation model more capable than Qwen, Claude, or ChatGPT by simply training another model.

That realization initially made the field feel inaccessible.

The idea behind MIRA came from an ordinary experience at my day job. My Claude coding session ended, and rather than wait, I connected a DeepSeek model to my VS Code Copilot harness. It worked, but it needed much more explicit direction than I expected. The following day, I connected the same model to a Claude-based coding harness and noticed a significant improvement in autonomy and coding performance, although it also consumed more tokens.

I could not prove exactly which internal mechanism caused the difference, but it led me to an important question:

Is the environment surrounding a model sometimes as important as the model itself?

The model had not changed. What changed was the harness: the instructions it received, how repository context was prepared, how tools were exposed, what prior information remained available, and how that information was retrieved.

I initially wanted to build a complete AI harness that could understand and operate across my desktop. That remains an ambitious long-term goal, but it also introduced an enormous scope and serious security risks.

Then I discovered the Qwen Cloud MemoryAgent track. Memory was at the centre of the question I had already begun asking. The difference between capable agent environments is not only the model they use; it is also how they preserve decisions, manage changing information, retrieve context, and construct the model’s next prompt.

Unlike foundation-model training, this problem operates largely at the context and memory-system level. It is financially accessible to an independent builder, immediately useful, and still open to meaningful architectural contributions.

I began studying agent-memory research using my structured paper-reading repository, which was forked from Shichun Liu’s Agent Memory Paper List. I moved from Generative Agents and CoALA to MemGPT, GraphRAG, temporal graph systems, reflection, decay-aware retrieval, foresight, and newer memory operating-system research.

The papers taught me that agent memory was not one isolated problem. Different systems addressed different parts of it: retrieval, reflection, temporal knowledge, graph reasoning, context management, prospective memory, or forgetting. Using CoALA as a broad yardstick, I began selecting the ideas that seemed most important for a practical memory system.

The difficult part was not collecting features. It was making them coexist.

Some approaches used incompatible graph structures. Some treated the context window as the memory itself. Some focused on durable facts but could not apply a correction immediately within the current session. Others generated useful abstractions but did not make their evidence inspectable.

Reconciling those tensions led to a distinct architecture—and eventually to MIRA. The paper formalized this around two consolidation timelines: immediate session continuity and slower cross-session learning, joined during prompt construction.

What MIRA Does

MIRA—Memory-Integrated Reasoning Architecture—is a persistent memory infrastructure layer for AI agents.

It does not attempt to improve an agent by continuously enlarging its prompt. Instead, it records conversation events, converts useful information into structured memory, tracks how that information changes, and retrieves only what is relevant to the current request.

MIRA can:

  • Carry goals, constraints, corrections, decisions, and unresolved questions through the current session.
  • Preserve useful information across separate conversations.
  • Distinguish an accepted change from an unresolved contradiction.
  • Retain outdated information as history without allowing it to control current answers.
  • Retrieve direct facts, relationships, and broader patterns through different retrieval modes.
  • Track future-relevant commitments and constraints through foresight records.
  • Build each prompt under a token budget.
  • Show the routing decision, retrieved records, and evidence that influenced an answer.

For example, when a user says that a project uses MongoDB and later corrects this to PostgreSQL, MIRA does not merely overwrite the old statement. It immediately applies the correction within the session, later records the change as a SUPERSEDED_BY relationship, treats PostgreSQL as current, and retains MongoDB as inspectable history.

How We Built It

MIRA is built around a fast interactive path, a slower memory-consolidation path, and a retrieval-gated prompt builder.

Immediate session continuity

Every user turn is first persisted as a raw observation. A lightweight session micro-path then extracts only information that may need to affect the next answer immediately:

  • Current goals
  • Active constraints
  • Explicit corrections
  • Decisions
  • Open questions
  • Resolutions

These provisional items enter a Session Working Set. A deterministic validator checks their structure, allowed scope, source evidence, type, and priority before they are allowed to affect prompt construction. This lets a correction such as “Use 2026, not 2025” influence the next response without waiting for the background worker.

Durable cross-session learning

A background slow path processes stored observations into longer-lived memory structures:

  • Atomic facts
  • Entities and graph relationships
  • Contradictions and supersessions
  • Reflections
  • Foresight records
  • Community summaries
  • Hot, warm, and cold memory tiers

SQLite is the canonical source of truth. ChromaDB serves as a rebuildable vector index, while graph nodes and edges are persisted in SQLite and projected into graph libraries when traversal or community algorithms are needed.

Retrieval and prompt construction

MIRA provides several retrieval behaviours:

  • Quick Mode for direct facts
  • Relational Mode for change, conflict, dependency, causality, and entity relationships
  • Deep Mode for patterns synthesized from reflections and community summaries
  • Auto Mode for selecting the appropriate route

The current router uses a hybrid strategy: deterministic rules handle clear queries cheaply, while an LLM classifier is used only for low-confidence or ambiguous routes when a provider is available.

Quick retrieval is also structured-first. Validated atomic facts, reflections, and foresight records receive priority over verbose raw observations. The original observations remain available as evidence and fallback rather than becoming the primary answer surface.

Contradiction detection follows a similar hybrid principle. Exact canonical matches are handled deterministically. Less obvious candidates are shortlisted through embedding similarity and verified in one structured LLM call. This was more reliable than the earlier pure heuristics, which struggled when extraction produced variations such as “Project” and “project deadline.”

Finally, the prompt builder merges the current message, recent turns, Session Working Set, durable hot memory, retrieved records, and ambient context. Lower-priority memories and older turns are trimmed first when the token budget is exceeded. Critical corrections, safety instructions, and the current request are protected.

MIRA is exposed through a FastAPI backend, a web application, an inspection interface, Slack, and an authenticated MCP service. It also includes GitHub OAuth, workspace isolation, Docker deployment, evaluation tooling, and answer traces.

Challenges We Faced

Turning separate research ideas into one coherent system

The hardest challenge was ideation.

It was easier to find useful memory mechanisms than to decide where each one belonged, when it should execute, what information it should consume, and how it should interact with every other layer.

For example, session continuity and durable memory appear similar at first, but they operate on different timelines. A correction may need to affect the next answer immediately, while durable confirmation may require more evidence and slower processing. Treating them as the same mechanism either introduces latency or promotes temporary information too aggressively.

We therefore separated provisional session state from confirmed cross-session memory and reconciled them during prompt construction.

Discovering the gap between architecture and runtime behaviour

Some mechanisms appeared correct in theory and even had passing tests, but evaluation showed that they were not contributing in practice.

Early ablation results suggested that reflection was unnecessary. Investigation revealed the opposite: reflection was gated on observations within one processing batch, while interactive evaluation processed observations individually. The layer was not weak—it was dormant.

Further inspection showed that reflection importance scoring favoured urgent words rather than stable user traits, reflections were not embedded into the vector store, and contradiction matching failed when fact extraction fragmented entities.

These issues were fixed only because the evaluation checked internal mechanisms rather than trusting the final answer.

Balancing deterministic logic with model judgement

Using an LLM for every routing, extraction, comparison, and validation decision would improve flexibility but increase latency, token usage, cost, and dependence on provider availability.

Making everything deterministic would be cheaper and easier to test, but brittle when language becomes ambiguous.

Our practical answer was not to choose one side universally. We used deterministic logic for clear, structural, and safety-sensitive operations, then introduced tightly gated LLM escalation where semantic judgement genuinely improved the result.

Evaluation cost and provider limitations

We built a LongMemEval-compatible benchmark pipeline, but we have not yet completed a full-scale benchmark run.

The complete benchmark requires many long-context model and judge calls. Token cost, execution time, and free-tier Qwen rate limits made it impractical to run the full dataset during the hackathon window. The current implementation includes the prepared dataset, cost estimation, resumable benchmark tooling, and a small official-protocol subset run.

That subset is evidence that the benchmark pipeline works, but it is not enough to claim broad benchmark performance. The current recorded run covers five temporal-reasoning examples with a 0.4 judge pass rate. Completing and improving the full benchmark remains future work.

Preparing for production-scale infrastructure

MIRA currently runs effectively as a single-server, production-minded prototype. It has authentication, workspace isolation, an API, a worker, containerized deployment, rate limiting, interfaces, and observability.

However, its local rate limiter is in-process. A genuinely distributed deployment with multiple API and worker instances would require shared infrastructure such as Redis or an edge-level rate limiter.

The number of interacting components also created a risk of technical debt. We managed this with clear module boundaries, typed repository functions, architecture decision records, isolated tests, static analysis, security checks, and mechanism-level evaluation.

Accomplishments We’re Proud Of

We are proud that MIRA moved beyond a paper architecture into a working end-to-end memory system.

The saved local evaluation currently passes 13 of 13 behavioural cases, covering direct recall, session corrections, cross-session memory, supersession, contradiction handling, foresight, reflection, community summaries, routing, and retrieval sufficiency.

The full architecture also passes 7 of 7 targeted ablation cases. Removing individual components causes the corresponding behaviour to fail. The vector-only baseline passes 2 of 7 cases, while the raw full-transcript baseline passes only 1 of 7. Because the ablation set is deliberately small, these numbers should be treated as directional evidence rather than a definitive statistical result.

More importantly, the evaluation does not check only whether the final answer contains a keyword. It verifies whether the intended retrieval mode ran, whether a Session Working Set item entered the prompt, whether a reflection was actually retrieved, and whether the correct graph edge was created.

We are also proud that MIRA exposes its internal process. Each answer can include the selected route, whether memory was used, the retrieved record types, their identifiers, and the evidence path that shaped the response.

What We Learned

Technical lessons

We learned that persistent memory is not primarily a storage problem. Saving information is the easy part. The harder problem is deciding:

  • What deserves immediate influence
  • What should become durable
  • What is still valid
  • What has been replaced
  • What conflicts with something else
  • What should be retrieved for this particular request
  • What should fit into the next prompt

We also learned that different stages need different levels of intelligence. Deterministic systems are valuable for validation, reproducibility, safety boundaries, and cheap obvious decisions. LLM judgement is most useful when it is constrained to ambiguous semantic questions with structured outputs and confidence gates.

Another major lesson was that observability is part of correctness. Without retrieval traces, graph inspection, queue health, and mechanism-level evaluation, a system can produce the right answer for the wrong reason.

Product lessons

From a product perspective, memory should not feel like invisible magic.

Users need to understand what the agent remembers, correct information that is wrong, distinguish current facts from historical ones, inspect what influenced an answer, and reset their data when necessary.

This changed MIRA from an architectural experiment into a usable system. We added workspaces, authentication, a web interface, memory inspection, retrieval traces, evaluation results, Slack support, MCP access, and workspace reset controls.

We also learned that powerful memory must remain selective. Remembering everything permanently is not automatically helpful. A useful product needs expiry, supersession, relevance filtering, scope boundaries, and user control—not merely more stored data.

Research lessons

Research taught us that combining ideas from multiple papers is not the same as designing an architecture.

Each paper carries assumptions about its storage model, timing, workload, graph structure, or evaluation environment. Two individually useful ideas may conflict when placed in the same runtime.

The process therefore required interpretation: deciding which ideas were fundamental, which were implementation details, which could coexist, and which needed redesign.

Evaluation also changed how we understood research evidence. A passing result does not necessarily prove that a component works. It may mean that the test never exercised it, another layer compensated for it, or the component never ran at all.

That was perhaps the most important lesson from MIRA:

A memory architecture should not be judged only by whether an agent eventually says the correct words, but by whether the intended memory mechanism actually produced them.

What’s Next for MIRA

MIRA is already a working, production-minded prototype, but several areas remain before it can become the broader agent infrastructure originally imagined.

The next steps include:

  • Running and improving the complete LongMemEval and LoCoMo-style evaluations.
  • Adding shared queue and rate-limit infrastructure for multi-instance deployment.
  • Strengthening worker observability and failure recovery.
  • Expanding user-controlled forgetting and deletion semantics.
  • Improving source-reliability modelling.
  • Investigating retrieval policies that learn from outcomes rather than remaining fixed.
  • Supporting multimodal memory and more advanced multi-user policies.
  • Connecting MIRA to richer agent environments, including the longer-term vision of a secure desktop-aware harness.

MIRA began with a question about why the same model behaved differently inside two different coding environments.

It has grown into a broader answer: model capability matters, but capability alone is not enough. An agent also needs an environment that can preserve experience, apply corrections, retrieve evidence, manage limited context, and explain what shaped its decisions.

That is the role MIRA is being built to serve.

Built With

Share this project:

Updates