Universal Agent OS - Qwen Cloud MemoryAgent

Persistent governance memory for AI coding agents. Four memory pillars with hybrid BM25+semantic recall, decay policies, and mistake prevention — powered by Qwen Cloud.

Comment

Inspiration

A developer uses AI coding agents daily. Over three months, they explain the same preferences dozens of times: preferred test framework, code style, architectural constraints, and past mistakes to avoid. Each new session starts from zero. Universal Agent OS was created to make memory a first-class safety and productivity layer for coding agents.

This project is derived from Universal-Agent-OS, our open-source agent governance framework. For this hackathon, we rebuilt the memory system around Qwen Cloud's embedding and chat APIs, adding hybrid BM25+semantic recall, self-reflection extraction, decay policies, mistake prevention, MCP server integration, and a multi-agent memory bus.

What it does

Universal Agent OS stores four memory pillars: State Memory, Your Persona, Minefield History, and Code Soul. Before responding, the backend builds a scored memory context using hybrid keyword + semantic recall powered by Qwen Cloud text-embedding-v3 embeddings, then sends it to a Qwen Cloud model. This helps the agent remember user preferences, preserve architecture intent, and avoid repeating known mistakes.

Key capabilities:

  • Hybrid recall scoring: BM25 + Qwen Cloud embedding similarity + freshness decay + importance weights, within a fixed context budget
  • Self-reflection extraction: Automatically extracts candidate memories from assistant responses and classifies them into the four pillars
  • Decay and forgetting: Old, low-importance memories decay over time while critical Minefield and Code Soul rules are never forgotten
  • Mistake prevention: Evaluates proposed actions against minefield memories and returns CLEAR, CAUTION, or BLOCKED verdicts
  • MCP Server: 6 tools for agent interoperability
  • Multi-agent memory sharing: Centralized memory bus for cross-agent learning
  • Qwen Cloud tool calling: Automatic memory operations via function calling API

How we built it

  • Backend: Python FastAPI with 30+ endpoints
  • Memory Store: Local JSON-backed four-pillar store with hybrid retrieval
  • Qwen Cloud Integration: OpenAI-compatible API path for chat completions and text-embedding-v3 for semantic recall
  • Frontend: Static HTML/JS workbench with tabbed inspector, recall trace, timeline, and memory controls
  • Testing: 116 automated tests (all passing)
  • Deployment: Deployed live on Alibaba Cloud ECS (Singapore) at 47.84.233.116 with Docker.

Challenges we ran into

Making memory visible and measurable was the biggest challenge. A MemoryAgent demo should show what memory was selected, why it was selected, what was forgotten, and how the response improved compared with a stateless model call. Balancing recall quality with context budget constraints required careful tuning of the BM25+semantic scoring weights.

Accomplishments that we're proud of

  • Built a complete memory governance layer with hybrid retrieval in under two weeks
  • Implemented self-reflection extraction that learns from the agent's own responses
  • Created a mistake prevention system that demonstrably blocks known bad actions
  • 116 tests covering all features
  • Frontend workbench that makes memory behavior visible without explanation

What we learned

  1. Memory is not just storage — scoring, budgeting, and forgetting are equally important
  2. Explainability matters — showing which memories were selected and why is more convincing than a black-box response
  3. Mock/real mode separation is essential for honest development
  4. Start with the demo — designing the frontend first keeps the project focused on what judges see

What's next for Universal Agent OS

  • Migrate memory store from local JSON to Redis or Alibaba Cloud TableStore
  • Add multi-session learning with cross-session preference tracking
  • Expand multi-agent memory bus for team-wide agent governance
  • Publish as an open-source framework for any LLM provider

Alibaba Cloud & Qwen Cloud Proof

We have strictly met the hackathon's deployment requirements:

  • Alibaba Cloud ECS: The backend is deployed live on a Singapore ECS instance.
  • Public URL: http://47.84.233.116:8000
  • Health Check: http://47.84.233.116:8000/health confirms qwen_mode: "real" and embedding_mode: "real".
  • Deployment Proof: http://47.84.233.116:8000/api/deployment/proof confirms status: "configured" and ECS runtime without exposing secrets.
  • Qwen API: We use DashScope (dashscope-intl.aliyuncs.com) for both qwen-plus chat completions and text-embedding-v3 (1024 dimensions) semantic recall.

Note on Demo Media: Some parts of the demo video and screenshots in the gallery were recorded during earlier local offline development tests (hence the "Offline" status visible in some UI frames). However, the system is fully deployed, online, and live on Alibaba Cloud for judging via the endpoints above.

Built With

Share this project:

Updates