RECALLOS

A privacy-first persistent AI memory system that remembers across conversations, retrieves relevant context, and protects sensitive data through local detection, secure re-hydration.

Comment

Inspiration

AI assistants are becoming increasingly capable, but they still have two fundamental problems: they forget and they expose.

Every new conversation often starts from zero. Users repeatedly explain their preferences, projects, goals, and previous decisions. At the same time, giving AI access to personal context means sending potentially sensitive information to cloud-based models.

We wanted to solve both problems together.

Instead of building another chatbot, we envisioned RecallOS as a persistent memory and privacy layer for personal AI — an assistant that remembers what matters, understands context over time, and keeps users in control of their information.

What it does

RecallOS is a privacy-first personal AI system with persistent, structured memory and intelligent model orchestration.

Its key capabilities include:

Persistent Memory: Automatically extracts meaningful facts, preferences, goals, and decisions from conversations and stores them across sessions and projects.

Semantic Retrieval: Recalls relevant information using semantic search and provides source-grounded evidence so users can understand where an answer came from.

Privacy Shield: Detects sensitive information locally and replaces it with deterministic tokens before sending requests to cloud AI models. Original information is restored locally when the response returns.

Intelligent Model Routing: Dynamically routes simple tasks to NVIDIA Nemotron Nano/Super and more complex reasoning tasks to Nemotron 3 Ultra through Nebius Token Factory.

Real-Time Privacy Audit: Allows users to inspect original prompts, sanitized cloud-bound requests, model allocation, latency, and token usage.

Human-in-the-Loop Controls: Requires explicit user approval before consequential agent actions are executed.

Our goal is to make AI genuinely personal without sacrificing privacy or user control.

How we built it

We designed RecallOS around a local-first security architecture, separating sensitive data processing from cloud-based intelligence.

The frontend uses Next.js, React, and Tailwind CSS to provide a unified dashboard, AI chat interface, memory center, project workspace, and privacy audit panel.

The backend is built with Python and FastAPI, coordinating conversations, memory retrieval, model selection, and agent operations.

The privacy engine handles local PII detection, deterministic tokenization, and response re-hydration. Sensitive information stays within the local security boundary, while sanitized requests are sent to external models.

For memory, we designed a structured persistence layer using SQLite for conversation data and metadata, alongside local vector storage using ChromaDB/Qdrant for semantic retrieval.

We integrated Nebius Token Factory to access NVIDIA Nemotron models and designed an intent and complexity-based routing system to select the appropriate model for different tasks.

We also incorporated safety boundaries and approval mechanisms to prevent agents from performing consequential actions without user authorization.

Finally, we designed an evaluation suite to examine retrieval quality, latency, and privacy leakage.

Challenges we ran into

One of our biggest challenges was balancing persistent memory with privacy. Storing everything is easy, but deciding what should be remembered, what should remain private, and what should be retrieved later is much harder.

We also faced several engineering challenges:

Designing deterministic tokenization and re-hydration without corrupting the original meaning of conversations.

Building selective memory retrieval instead of simply injecting entire conversation histories into every prompt.

Handling outdated or conflicting memories when user preferences and project information change.

Routing requests between different AI models without introducing unnecessary latency or complexity.

Integrating multiple services while maintaining a consistent local-first security boundary.

Making privacy visible and understandable rather than treating it as an invisible backend feature.

These challenges pushed us to think beyond simply connecting an LLM to a database and focus on the complete AI system architecture.

Accomplishments that we're proud of

We are particularly proud of bringing together several ideas that are often treated as separate engineering problems.

Designing a unified architecture combining persistent AI memory, privacy protection, and intelligent model orchestration.

Building a deterministic privacy layer that allows sensitive information to be masked before cloud transmission and restored locally.

Designing a real-time privacy audit interface that makes data handling observable.

Implementing a memory architecture focused on selective extraction, semantic retrieval, and conflict resolution.

Integrating NVIDIA Nemotron models through Nebius Token Factory for different levels of reasoning.

Designing human approval mechanisms for controlled agent execution.

Most importantly, we transformed an idea about AI that remembers into a concrete system architecture with distinct, independently testable components.

What we learned

This project taught us that building useful AI products requires much more than selecting a powerful language model.

We learned about memory extraction, vector databases, semantic retrieval, privacy-preserving data processing, model orchestration, and agent safety.

We also realized that persistent memory is not simply about storing more information. It is about identifying relevant information, maintaining its validity, retrieving it at the right moment, and providing evidence for the context being used.

Another important lesson was that privacy should be part of the architecture from the beginning, not something added after the AI system is already built.

What's next for RECALLOS

We envision RecallOS becoming a universal memory and privacy infrastructure for AI assistants and autonomous agents.

Our next steps include:

Expanding integrations with MCP-compatible AI assistants, development tools, and autonomous agents.

Improving automatic memory extraction, consolidation, and long-term conflict resolution.

Exploring more advanced retrieval techniques combining semantic, keyword, and graph-based relationships.

Strengthening local privacy detection and expanding coverage for sensitive information.

Improving model routing through continuous evaluation of reasoning quality, latency, and cost.

Developing more comprehensive benchmarks for memory accuracy, retrieval performance, and privacy guarantees.

Giving users greater control over memory inspection, editing, deletion, and portability.+

Built With

Share this project:

Updates

Submission history