ARIA Memory Stack

Inspiration

Most AI assistants suffer from a fundamental architectural limitation: they have no memory beyond the current session. Every conversation begins from zero — no accumulated context, no recognition of past decisions, no understanding of the person behind the request. This is not a minor inconvenience. It is a ceiling that prevents AI from becoming genuinely useful over time.

The ARIA Memory Stack was built to remove that ceiling. The inspiration came from observing how human memory actually works — not as a flat log of events, but as a layered system where emotional weight, relationship context, and pattern recognition all contribute to what gets retained, surfaced, and acted upon. The goal was to replicate that architecture in software.

What it does

ARIA Memory Stack is a multi-layer persistent memory infrastructure for AI agents, powered by Qwen Cloud. It provides:

• Semantic memory — vector-based retrieval via Qdrant (768-dimensional embeddings) that surfaces contextually relevant memories based on meaning, not keyword matching.
• Graph memory — a PostgreSQL relationship graph that connects people, events, decisions, and facts across time.
• Emotional context layer — each memory is tagged with an emotional weight (neutral, positive, negative, important, critical), enabling the AI to prioritize what matters most to the user.
• Cross-session identity — persistent recall across multiple channels simultaneously: web interface, Telegram, and WhatsApp, all sharing the same memory layer.
• Memory Optimizer — an automated pipeline for deduplication, confidence scoring, and memory decay that keeps the knowledge base clean and relevant over time.
• MultiBrain model router — dynamically selects the optimal Qwen model based on task complexity, routing between qwen3-235b, qwen-max, qwen-plus, qwen-turbo, qwen3-32b, and qwen2.5-72b.

The system was validated through a live benchmark comparing memory recall performance across six major AI agents — 50 structured questions, 300 API calls, evaluated by an LLM-as-judge (Gemini 2.5 Flash).

How we built it

The architecture is composed of six integrated layers, all self-hosted on a Linux VPS and connected through an internal MCP Gateway with 24 tools:

1. Base UserMemory — primary source of truth, storing 248+ structured memories with type classification, emotional weight, subject tagging, and confidence scores.
2. Memory Bus (port 4055) — cron-triggered export every 6 hours, syncing memories from Base into the vector and graph databases.
3. Qdrant vector database (Docker, port 6333) — 643 deduplicated vector points, each with 768-dimensional embeddings for semantic retrieval.
4. Unified Memory (PostgreSQL aria_graph, port 4050) — 242 memories with full embeddings, accessible via a Flask REST API.
5. MultiBrain v2.1 (port 4040) — the Qwen Cloud API router. Selects the optimal model per task using a routing table based on task type and complexity. Integrates six Qwen Cloud models via the DashScope International endpoint.
6. Memory Optimizer v2.1 (port 4020) — automated deduplication with safety limits, null-confidence handling, and merge constraints to prevent data loss.

The MCP Gateway exposes shell execution, RAG search, graph queries, file editing via Hashline, and YouTube transcript extraction as callable tools within the agent pipeline.

Challenges we ran into

The most critical incident during development was a mass-deletion event caused by the Memory Optimizer. A combination of null confidence values and empty subject fields caused the optimizer to classify 602 out of 619 memories as stale or duplicate, deleting them within two cycles in under ten minutes. Recovery required a full re-export from Base, four targeted safety fixes to the optimizer logic, and a complete re-sync of the pipeline.

Cross-session recall proved to be the hardest cognitive challenge to solve at the architecture level. Getting an agent to correctly retrieve something said three weeks earlier, in a different channel, in a different context, without hallucinating or confusing similar entries, required careful tuning of both the embedding pipeline and the retrieval scoring logic.

Privacy enforcement across multiple isolated users was another significant challenge. The system must guarantee zero cross-contamination between user memory silos at the platform level, not just at the application level.

Accomplishments that we are proud of

• Designed and deployed a complete persistent memory infrastructure from scratch within 30 days, running live in production across three communication channels.
• Built and executed the first systematic benchmark comparing memory recall performance across six major AI agents under equal context conditions, with LLM-as-judge evaluation.
• Developed MultiBrain — a model router that dynamically selects the optimal Qwen model per task type, reducing unnecessary token consumption while maintaining response quality.
• Demonstrated that emotional context tagging produces measurably better recall relevance than pure vector similarity alone.
• Recovered from a catastrophic data loss incident with zero permanent memory loss and implemented architectural safeguards to prevent recurrence.

What we learned

Memory is not storage — it is curation. The hardest problem is not saving data. It is deciding what to keep, what to decay, what to merge, and what to weight as emotionally significant. Human memory performs this automatically through biological mechanisms. Replicating it in software requires explicit architectural decisions at every layer.

The benchmark results confirmed what the architecture suggested: single-hop recall is largely a solved problem across all major models. Cross-session recall, contradiction detection, and privacy enforcement are where the real gaps remain — and where purpose-built memory infrastructure like ARIA Memory Stack creates the most measurable value.

Qwen models demonstrated strong performance on structured memory retrieval tasks, particularly qwen3-235b on complex multi-hop reasoning across layered memory contexts.

What is next for ARIA Memory Stack

• Phase 2 benchmark — test the full ARIA stack (RAG + graph + emotional weight + cross-session) against raw LLM reasoning to quantify the real advantage of purpose-built memory infrastructure.
• Memory federation — allow multiple ARIA instances to share selective memory pools while maintaining strict per-user isolation.
• Proactive memory surfacing — surface relevant past context before the user asks, based on pattern detection and scheduled triggers.
• Open source release — package the Memory Stack as a standalone, model-agnostic library that any developer can integrate into their AI agent pipeline.
• Native mobile application — a SwiftUI iOS app with full persistent ARIA memory on mobile, currently in active development.

Built With

• cron
• docker
• flask
• javascript
• linux-(vps)
• mcp-gateway
• nginx
• node.js
• pm2
• postgresql
• python
• qdrant
• qwen-cloud-api-(alibaba-cloud-dashscope-international)
• qwen-max
• qwen-plus
• qwen-turbo
• qwen2.5-72b-instruct
• qwen3-235b-a22b
• qwen3-32b
• rag-(retrieval-augmented-generation)
• react
• tailwind-css
• telegram-bot-api
• vector-embeddings-(768-dimensional)
• whatsapp-baileys