Rice Memory Agent — Inspiration & Retrospective

Inspiration

LLMs forget everything the moment a session ends, and they choke once the context window fills up. We wanted an agent that remembers across sessions the way a computer holds state across its memory hierarchy — hot and fast for what matters, cold but retrievable for the rest.

What it does

You drop in your writings (PDF/TXT) and the agent reads, chats about, and learns from them — keeping the important bits in fast memory and quietly archiving the rest without ever losing it.

How we built it

A FastAPI backend with a SQLite + sqlite_vec store, lightweight in-repo memory controllers (no heavy framework), a pluggable tool layer, a React frontend behind nginx, and a systemd unit so it runs as a real supervised service.

Challenges we ran into

The context window kept overflowing, PDF uploads hung because we reloaded the embedding model on every request, and chunk metadata leaked into the UI — all fixed with hard budgets, a cached embedder, and a strict response contract.

Accomplishments that we're proud of

It survives restarts and resumes mid-conversation from a single file, the CPU-pyramid memory model actually works in practice, and it deploys like a proper service rather than a prototype.

What we learned

Bounded context is a design problem, not a prompt trick; forgetting is safer as demotion than deletion; and per-request model loading will quietly ruin your latency.

What's next for Rice Memory Agent

Smarter section-aware chunking, multi-modal PDF ingest, a UI to edit memories directly, and an eval harness to measure how well the tiers really perform

Built With

• dashscope
• python
• qwen
• react