-
-
Hybrid retrieval: dense embeddings plus BM25 fused into one score, catches paraphrases keyword search alone misses
-
The project's own knowledge graph, tickets, decisions, and notes all linked together and browsable in VS Code
-
How a fact moves through the system, written, promoted to hot cache, then archived from disuse or invalidated if contradicted
-
Full system, client to backend to three-tier memory, with Qwen running the chat and the embeddings
Inspiration
It was around 4am and I was doom scrolling Instagram like always, and this reel popped up of some guy hyping himself up for building "an AI agent" using a no code platform. Got me thinking about how easy it's become to slap something together without actually knowing what's happening under the hood.
I've been an Iron Man fan since I was a kid, honestly JARVIS is probably why half of us got into this field in the first place. So I wanted to find out for myself what it actually feels like to build something like that from scratch. No templates, no drag and drop builder doing the thinking for me. Just me trying to actually understand memory, tool calling, and what it means for an agent to remember something instead of just responding to it and forgetting the second the chat closes. That's where Pi came from.
What it does
Pi is a personal agent built around one bet: memory is the product, not a feature you bolt on later. Most "memory" in AI agents is just context stuffing, shove the last 20 messages back into the prompt and call it a day. Pi runs an actual three tier write, read, and forget lifecycle instead:
L1, raw log: every conversation turn, archived and pruned after about 30 days. L2, durable facts: at the end of a session, Qwen distills the conversation into discrete facts with importance scores and categories. Semantic dedup, using Qwen's text-embedding-v4 vectors, stops the same fact from piling up as five slightly different paraphrases of itself. L3, hot cache: the facts that actually matter right now, injected into the system prompt every turn under a hard token budget. Recalling the right thing inside a limited context window is the real design constraint here, not something I bolted on after. Retrieval is hybrid and query aware. Every message gets embedded through Qwen and fused, dense cosine similarity plus BM25 plus importance and recency boosts, across L3 and L2. The clearest proof case in my test suite: I store "the lab uses zebrafish as the model organism," then in a completely fresh process I ask "which animal is the experiment on now?" Zero shared keywords. Plain keyword search finds nothing. The hybrid retriever still surfaces it.
Forgetting is a real feature, not a bug I'm ignoring. Facts can carry an expiry and get pruned by a retention policy, unused facts decay on their own the longer nobody touches them, and contradicted facts get invalidated instead of deleted, so "what did I tell you before?" still gets answered honestly instead of the fact just vanishing like it never happened.
There's also a live dashboard at /memory that shows all of this happening in real time instead of asking you to take my word for it. Hot L3 rows ranked by importance, a query box that shows the actual fused retrieval score for anything you type, and a color coded forgetting ledger that says exactly what got forgotten, when, and why.
How I built it
Python, stdlib first as much as I could get away with. FastAPI brain server with SSE streaming and bearer auth, SQLite for the hot memory tier, Supabase for the durable tiers, BM25 plus in process cosine similarity for retrieval (no vector database, I don't have the scale to justify one yet). Qwen's qwen3.7-max sits first in every routing tier through the DashScope OpenAI compatible endpoint. It runs the actual conversation, the session summaries, and the L2 fact distillation. text-embedding-v4 handles embeddings for both the retriever and the dedup step. The router enforces a per provider daily token budget with automatic brownout and failover, because hackathon credits are finite and I really did not want to wake up to a runaway loop that burned through everything overnight. The backend runs on an Alibaba Cloud ECS instance under systemd. Alibaba Cloud proof of use lives in core/providers/qwen.py (chat) and memory/semantic_dedup.py (embeddings), both hit the DashScope endpoint directly.
Every change ships behind a CI gate, 310+ files, 160+ test files, and a set of memory round trip contract tests that assert write and read stay consistent for every tier, because write/read divergence is the bug class that actually kills memory systems and I've been burned by it more than once building this.
Challenges I ran into
Where do I even start. Memory perfection is basically impossible, and I ran into memory leakage, duplication, and a forgetting problem all at once early on. One bug that stuck with me: I told Pi I prefer coin trading, and after that literally any generic "look up stocks for me" request would come back with Bitcoin and Ethereum prices, because the preference had bled into context it had no business touching. Had to actually go build a proper scoping and contradiction system so a stored preference doesn't just override every future request that's vaguely in the same neighborhood.
Early on Pi was basically just raw API calls with zero context, no memory of anything between calls. Building the actual "bubble," the context window that lets Pi hold onto what's relevant across a conversation instead of treating every message like the first one, is what actually made Pi feel like Pi instead of a stateless chatbot with a nice name.
Switching between modes broke things too. When I switched from normie mode (fast, cheap, for quick chat) to root mode (the full tool loop), Pi would lose most of its context in the handoff, like talking to two different people who don't talk to each other. I had to build a proper conversation coherence layer so context actually survives a mode switch instead of getting dropped on the floor.
On the retrieval side, paraphrase recall was its own wall. Plain keyword search silently fails the second someone rewords a question, so I added embeddings on the hot tier and had to fuse the scores without slowing down the live write path (embeddings backfill at session exit instead of inline, so a user is never sitting there waiting on an embedding call just to save a message). And honest forgetting turned out to be harder than it sounds: delete a contradicted fact outright and you break "what did I say before," keep it around and it poisons future recall. Invalidating instead of deleting, so it is still queryable but clearly marked as superseded, fixed both problems at once.
Accomplishments that I am proud of
A test that actually reproduces the paraphrase gap and proves the fix instead of just asserting it works. 300+ closed engineering tickets with an append only solutions log, this thing runs its own build, test, ticket, verify loop on itself, it is not a demo I threw together for the weekend. And getting every memory operation, summarizing, distilling, embedding, running fully on Qwen with budget aware routing, instead of one token API call bolted on just to check a box for the hackathon.
What we learned
I'm a freshman in CS. Before this I'd only taken CSC 1301 and 1302, so I had basically zero idea how real projects actually get built. Pi is the single thing that taught me how the real world stack fits together, git and GitHub, actual databases, routing, CI gates, all of it, way more than any class has so far. And with how much AI coding assistance is everywhere right now, I learned the difference between vibecoding something into existence and actually understanding what you built well enough to use it properly. Anyone can get a feature to appear. Knowing why it works and when it'll break is the part that actually matters, and that's the part I was chasing this whole project.
On the technical side, wiring a new provider into a router that was already tuned around a different model's behavior taught me how much invisible tuning goes into making tool calling reliable in the first place. And deploying to Alibaba Cloud for the first time under a hard deadline taught me to test the live path early instead of just assuming whatever worked locally would carry over.
What's next for PI
Scheduled memory consolidation, something closer to actual sleep for the agent instead of only ever writing memory at session end. A retrieval quality eval harness so recall accuracy is actually measured instead of just assumed. And eventually letting Pi's own autonomous sprint runner close its own memory related tickets without me driving every single step. Longer term, every conversation I have with Pi right now is data toward something bigger: eventually training it on its own weights instead of always routing through someone else's API, so it stops being a wrapper around a provider and starts actually being its own thing, closer to what got me into this in the first place.
Log in or sign up for Devpost to join the conversation.