COSMOS-Q: A Cognitive Memory System for Qwen Agents

Inspiration

AI agents are stateless and forget everything when a chat session ends. Storing full chat histories is expensive and slows down response times. We built COSMOS-Q to give Qwen agents long-term, human-like memory. Instead of storing exact logs, it version-controls memories, forgets useless details, and summarizes experiences into high-level knowledge.

What it does

COSMOS-Q manages long-term memory across sessions:

• Updates old memories when new facts appear.
• Selects the most important memories to fit inside the prompt token limit.
• Fades out old, conflicting, or unused memories over time.
• Clusters similar experiences and summarizes them into structured facts (e.g., user preferences).

How we built it

We deployed the system on Alibaba Cloud (Singapore region):

• ECS: Runs the real-time FastAPI MCP server.
• ApsaraDB RDS PostgreSQL: Stores data and 1024-dimension embeddings using the pgvector extension.
• Function Compute: Runs the daily cleanup and consolidation tasks.
• Qwen Cloud APIs: Used for text embeddings, standard completions (with thinking mode for summary reasoning), and the Responses API (for server-side conversation history linking).

Challenges we ran into

• Database Permissions: The default RDS setup blocked our application user from loading the vector extension. We resolved this by natively creating the database under our application user's direct ownership.
• ECS Disk Exhaustion: The initial Docker image was 4GB due to local machine learning packages. We stripped out local models and routed embedding calculations to Qwen Cloud, shrinking the image to under 150MB.
• Serverless Size Limit: Because OSS uploads were disabled on our account, we had to fit our Function Compute package under 5MB. We replaced the heavy official SDKs with lightweight, pure-Python database drivers (pg8000) and native REST clients (urllib), bringing the final ZIP file down to 2.9MB.

Accomplishments that we're proud of

• Shrank the serverless ZIP package by 80% (down to 2.9MB) to deploy inline.
• Enabled Qwen's thinking mode inside the serverless handler to generate high-level, human-readable summaries of clustered memories.
• Built a fully automated deployment setup using Terraform and Serverless Devs.

What we learned

• Offloading embedding tasks to cloud APIs (like Qwen text-embedding-v3) is much faster and cheaper than running local models.
• Custom API wrappers and pure-Python drivers are highly effective tools for keeping serverless functions small and fast.
• Set up database ownership before running migrations to avoid cloud privilege conflicts.

What's next for Cosmos-Q

• Shared Agent Memory: Allowing multiple Qwen agents to read and write to the same central database.
• Edge Caching: Storing memory briefs closer to the agent to reduce latency.
• Custom Schema Types: Allowing agents to define new knowledge categories on the fly.

Built With

• alibaba-cloud
• qwen