EchoDesk - Memory Agent

Inspiration

Most AI assistants are only as good as their context window — the moment a conversation ends, everything about you disappears with it. As a solo founder or freelancer, you don't want to re-explain your business, your clients, or your deadlines every single session. We were inspired by how human memory actually works: we don't remember everything perfectly forever, and that's a feature, not a bug. Important things stick, related details get consolidated into summaries, and irrelevant small talk fades away. We wanted to build an assistant that mimics that — a persistent-memory "Chief of Staff" that remembers what matters across sessions, without brute-force stuffing the entire chat history into every prompt.

What it does

EchoDesk is a cross-session memory agent that:

• Encodes every fact, decision, deadline, and preference as an individual memory chunk with its own importance score, timestamp, and embedding.
• Retrieves only the most relevant memories under a strict 2K-token budget, ranked by semantic similarity, recency, importance, and decay — so the assistant stays fast and cheap even as memory grows.
• Decays and forgets low-value memories (like small talk) over simulated time, so stale noise never resurfaces.
• Consolidates related memories into higher-level summaries during a simulated "sleep" cycle, preserving source lineage back to the original facts.
• Proves it works with a built-in benchmark screen showing recall precision and average context tokens across fixed test queries — including a negative test that confirms decayed noise is correctly not recalled.

The demo walks through four steps: seed Session 1 with facts, jump to Session 2 days later and recall a specific detail, fast-forward time to trigger decay/consolidation, then run the benchmark.

How we built it

EchoDesk is a full-stack Next.js (App Router) application with:

• Frontend: React + Tailwind, including a custom "Memory Brain" visualization that shows memories as nodes that brighten on recall and dim as they decay or get consolidated.
• Backend: Next.js route handlers exposing a memory API contract (encode, retrieve, decay, consolidate).
• Database: PostgreSQL via Drizzle ORM, storing memories with importance scores, access counts, decay scores, and consolidation lineage.
• Reasoning/scoring: A Qwen-compatible adapter (DashScope API) for semantic scoring and embeddings, with a deterministic local fallback so the demo runs reliably even without API keys.
• Target deployment: Designed to run against a FastAPI backend on Alibaba Cloud ECS for the hackathon's infrastructure track.

Challenges we ran into

• Getting retrieval to feel "smart" under a hard token budget. Balancing semantic similarity against recency and importance so the right memory surfaces — without just recalling everything — took a lot of tuning.
• Making decay and consolidation visible, not just theoretical. It's easy to decay a number in a database; it's harder to make that decay legible to a person watching the demo, which is what drove the Memory Brain visualization.
• Environment and infrastructure friction. Wiring up Postgres, Drizzle, and environment variables correctly for both local development and cloud deployment surfaced more configuration issues than expected — a good reminder that infrastructure plumbing is often the real work behind a clean demo.

What we learned

Building EchoDesk reinforced that "memory" for an AI agent isn't just storage — it's a full lifecycle of encoding, scoring, retrieving, decaying, and consolidating, much like human memory. Getting each stage right, and making that lifecycle visible and provable (not just claimed), was the real engineering challenge behind the project.

Built With

• alibaba-cloud
• dashscope
• drizzle-orm
• fastapi
• node.js
• postgresql
• qwen
• react
• tailwindcss
• typescript