Graphy Awareness

A conversational agent that builds an explainable memory graph of you, forgets what fades, and justifies every recommendation it makes.

Comment

Inspiration

Every conversational agent we'd used forgot us the moment the session ended. Track 1 asked for a MemoryAgent that "makes increasingly accurate decisions across multi-turn, cross-session interactions" — but most memory implementations we'd seen just dump everything into a vector store and hope similarity search finds the right chunk. That's a black box: you can't explain why the agent recalled something, and you can't tell it to forget.

We wanted memory you could actually look at. So instead of embeddings, we modeled memory as a typed graph — Need, Fear, Preference, Trait, Fact, Entity nodes connected by relationships like satisfies and causes. The moment we added Entity nodes — concrete things like "running" or "public speaking" — the graph stopped being just a psychological profile and became something that could justify a real recommendation: this user fears X, and Y satisfies that fear, so suggest Y. That reasoning chain, visible and inspectable, is what we built the whole project around.

What it does

Graphy Awareness is a chat agent with two views:

  • Chat — talk to the agent; every exchange is analyzed and written into your personal memory graph, which redraws live next to the conversation.
  • Insights — the full graph as an interactive, force-directed visualization, next to a panel of AI-generated recommendations that each cite the exact nodes and edges that justify them (e.g. "the user's fear of public speaking is satisfied by running → suggesting a running club").

Memory is real, not simulated: sign in with your account (AWS Cognito), close the tab, come back later, and the agent still knows you — because the graph lives in Alibaba Cloud Tablestore, keyed to your verified identity, not a browser session.

How we built it

Backend — Python on Alibaba Cloud Function Compute, talking to Qwen (qwen-max) over its OpenAI-compatible endpoint. Each chat turn does two Qwen calls: one for the reply (streamed), one forced tool-call for structured memory extraction, so we get reliable JSON instead of parsing free text.

The three memory pillars are implemented as actual mechanisms, not slogans:

  • Efficient storage & retrieval — nodes and edges are individual rows in Tablestore (partitioned by user), not a JSON blob, so writes are atomic and reads are a single range-scan. Before every LLM call we rank nodes and only inject the top ones:

$$ \text{score} = 0.6 \cdot \text{intensity} + 0.3 \cdot \text{recency} + 0.1 \cdot \text{confidence} $$

  • Forgetting — intensity and confidence decay exponentially on every request:

$$ \text{intensity}_t = \text{intensity}_0 \cdot 0.5^{\,t / 30\text{ days}} $$

Nodes that fall below a threshold silently stop showing up in context. Contradictions get an explicit superseded_by edge instead of just averaging out, and users can hard-reset with a "Clear graph" button.

  • Recall under limited context — the prompt only ever receives the top-k ranked nodes, so context size stays bounded no matter how long the relationship with a user runs.

Frontend — Next.js 16 / React 19 on Vercel, with a hand-rolled d3-force + SVG graph visualization (drag, zoom, click-to-inspect) instead of a charting library, to avoid dependency-compatibility risk on a brand-new React version. Auth is AWS Cognito via react-oidc-context — user_id is the verified Cognito sub, not a generated id, which is what makes cross-session recall provable rather than just claimed. The client never talks to Function Compute directly: Next.js API routes proxy every call, verifying the Cognito access token server-side before forwarding, so the backend URL never appears in the browser and a user can't spoof another user's id even from devtools.

Challenges we ran into

Deploying to Function Compute was the fight of the hackathon. In order:

  1. The openai SDK depends on Rust-compiled pydantic-core/jiter, which meant we couldn't build a Linux-compatible package without Docker (which we didn't have installed). We rewrote the Qwen client on raw requests — pure Python, no compiled extensions, cross-platform by default.
  2. FC's custom.debian10 runtime shipped a Python old enough to choke on modern syntax. We switched to the managed python3.10 runtime.
  3. FC's Python HTTP invocation isn't WSGI — it calls handler(event, context) with a raw JSON payload, not a WSGI environ. We reverse-engineered the actual event shape by logging it in production and wrote a manual WSGI adapter so Flask could keep working unmodified.
  4. Two nasty frontend bugs only showed up after real use, not code review: a stale closure in our d3-force tick handler kept resetting the rendered graph to empty, and an impure setState updater (reading a mutable ref inside the updater function) crashed under React 19's dev-mode double-invocation — React was right to catch it. Both were only found by actually driving the app with Playwright and watching it break, not by reading the code.

Accomplishments that we're proud of

  • All three memory pillars are real, testable mechanisms — decay math, per-row Tablestore writes, ranked context injection — not just documented intentions.
  • We debugged Function Compute's Python HTTP contract from zero documentation by instrumenting production and reading the raw event, rather than giving up and downgrading to a simpler (less honest) architecture.
  • The graph is genuinely interactive and updates in real time — you watch nodes appear as you talk to the agent, which is the clearest possible proof that memory is being written, not just claimed.
  • We hardened the security model under time pressure instead of shipping with an exposed backend URL and spoofable user ids: real auth, real token verification, real BFF proxy.

What we learned

That "deploy to serverless" and "it works locally" are two different projects, especially on a platform (Alibaba Cloud FC) with less community documentation than AWS/GCP — a lot of our debugging was empirical (log everything, read the actual bytes) rather than reference-driven. We also relearned a lesson about React the hard way: state updater functions must be pure, and framework "gotchas" like double-invocation exist because real bugs hide behind impurity.

What's next for Graphy Awareness

  • Multi-hop graph reasoning for recommendations (2–3 hops instead of direct neighbors) as the graph grows denser.
  • Embedding-based retrieval combined with the ranking score, for users whose graphs get large enough that keyword/label matching starts missing things.
  • A domain-specific recommendation mode aimed at real advertising/personalization use cases, since the Entity node is already designed as the bridge from psychology to product.
  • Shared or team memory graphs, for use cases beyond a single user talking to a single agent.

Built With

Share this project:

Updates