Inspiration
Modern AI assistants are powerful, but their long-term intelligence is usually tied to one provider. Switching models often means losing accumulated context, while persistent personalization can require sending increasingly personal data to the cloud.
We started Portable Brain with a different question:
What if the model were replaceable, but the user's brain stayed theirs?
Our research covered long-term agent memory such as MemGPT, Generative Agents and MemoryBank, retrieval systems such as RAG and GraphRAG, privacy-preserving text sanitization, contextual user modelling, cryptographic deletion and local-first software.
This led to our core idea: separate persistent personal intelligence from the model using a user-owned memory and privacy layer.
What it does
Portable Brain maintains an encrypted long-term memory containing approved information about conversations, people, projects, events and preferences.
Instead of sending the whole memory to an LLM, it retrieves a small relevant subset. Conceptually:
$$ M^*=\arg\max_{M'\subseteq M}\sum_{m_i\in M'}U(m_i) $$
subject to:
$$ \sum_{m_i\in M'}Tokens(m_i)\leq B $$
where (B) is the model's context budget.
Our retrieval utility combines ideas from semantic retrieval and memory research:
$$ U(m)= w_sS_{semantic} +w_gS_{graph} +w_rS_{recency} +w_iS_{importance} $$
PBrain also places a privacy gateway before cloud inference. Private identities are replaced with stable placeholders such as Person_2, processed by the model, and restored locally afterward.
The longer-term privacy objective is:
$$ \max_{x'} \; Utility(x')-\lambda\,PrivacyRisk(x') $$
meaning we want to reveal enough information to solve the task without unnecessarily exposing identity.
PBrain also supports cryptographic forgetting. Each memory object can be encrypted using its own key:
$$ c_i=Enc_{K_i}(m_i) $$
Forgetting destroys (K_i). Old Git history may still contain ciphertext (c_i), but without the key it can no longer recover the forgotten memory.
How we built it
PBrain is currently a Python-based Personal AI system with an encrypted persistent brain, structured memories, graph relationships, conversational archives and configurable model providers.
After each conversation, the model does not directly modify memory. It proposes operations which the user can accept, edit, reject or merge.
This gives us:
Conversation
↓
Memory Extraction
↓
Entity / Duplicate Check
↓
Proposed Change
↓
User Review
↓
Encrypted Brain
Retrieval combines direct entity matching, semantic similarity, graph traversal, recent conversations and archived material under a fixed token budget.
We also built evaluation tooling to test memory continuity, privacy leakage and response quality instead of relying only on subjective demos.
Most recently, we added a dedicated research and experiment-record system so architectural decisions can be connected to experiments rather than only implementation intuition. This was merged into the repository as part of the research-system work.
Challenges we ran into
Several of our hardest problems only became visible after building the system.
One major bug came from our privacy architecture. New chat episodes were initially treated as local-only data. This meant that, when using a cloud model, the second turn could lose the first turn's conversational history and memory extraction could be blocked. We had to change the permission logic so a conversation could return only to the models that had already legitimately received it. This was fixed through the repository's release-check work.
Our evaluation system also exposed failures in the evaluators themselves. For example, one grader incorrectly rejected the response "I don't actually know", while another accepted a correction even though the assistant still referenced the wrong person. We had to redesign those checks around semantic intent rather than simple word matching.
Even terminal UX caused unexpected issues. Markdown formatting such as **bold** was appearing literally because the CLI displays raw text. We changed model instructions and, because prompts affect deterministic assembly checks, also had to update the corresponding golden assembly hash.
Privacy was another challenge. Replacing a person's name is not enough:
Name removed
+ rare project
+ university
+ location
= potentially identifiable person
Our research therefore pushed us beyond simple PII masking toward re-identification risk as the real problem.
Finally, deletion proved much harder than DELETE FROM memories. Git deliberately preserves history, so deleting the latest copy does not delete older versions. That led us to per-memory encryption and crypto-shredding instead.
Accomplishments that we're proud of
We moved beyond the standard:
LLM + vector database
architecture.
PBrain now combines:
- reviewed, evidence-backed persistent memory
- graph and semantic retrieval
- encrypted user-owned storage
- provider-independent model access
- privacy-preserving cloud inference
- cryptographic forgetting
- archived conversation retrieval
- automated privacy and behavioural evaluations
- a research/experiment system for testing architectural decisions
Most importantly, the system treats facts, AI interpretations and user-approved information differently instead of allowing model-generated assumptions to silently become permanent truth.
What we learned
The biggest lesson was that Personal AI is not primarily an LLM problem.
It is a combination of:
$$ \text{IR} +\text{Graph Systems} +\text{Cryptography} +\text{Privacy} +\text{HCI} +\text{ML} +\text{Distributed Systems} $$
We also learned that more memory is not necessarily better memory. Retrieval quality, provenance and forgetting are just as important as storage.
Finally, architectural guarantees are stronger than prompts. Asking an agent not to expose something is weaker than designing a gateway through which that information cannot pass unchanged.
What's next for Portable Brain
Our next research direction is to make PBrain adaptive without turning it into another opaque profiling system.
We want to model preferences probabilistically:
$$ P(preference\mid evidence) $$
rather than storing assumptions as facts, and use contextual bandits and workflow prediction to improve interaction over time.
We are also exploring:
- privacy-aware multi-model routing
- stronger re-identification protection
- temporal and hierarchical memory
- decision provenance and replay
- shared encrypted memories between trusted users
- social key recovery
- secure cross-device synchronization
The long-term goal is simple:
Models should be replaceable. Your personal intelligence should not be.