Theos

Theos: A scalable, Private, secure, and low-latency AI assistant framework featuring CockroachDB long-term memory, AWS Bedrock integration, and dynamic Model Context Protocol (MCP) tool execution.

Comment 2

Theos — Your Private, Open-Source Personal AI Agent

Inspiration

We asked ourselves a simple question: Why does using an AI assistant mean giving up control of your data?

Every major AI platform today operates on the same model — your conversations are logged, your data trains their models, and you have zero visibility into what happens to your prompts once they leave your browser. The terms of service are long, the opt-out mechanisms are buried, and the default is always to collect everything.

On top of that, these platforms give users almost no control over the agent itself. You can't configure the context window percentage, you can't limit how many tools the agent retrieves per message to control costs, you can't swap models without switching apps entirely, and you can't manage the agent's long-term memory without writing SQL. The agent is a black box — and an expensive one at that, because closed-source models charge premium per-token rates.

We said: these three problems — privacy, control, and cost — are all solvable. Not with a research prototype, but with a production-grade, open-source application that any user can deploy locally or in the cloud and own completely.

That's Theos.

What It Does

Theos is a fully self-hosted, privacy-first AI agent platform built on CockroachDB and AWS. It gives users complete ownership over a personal AI assistant — from the model it runs, to the tools it can access, to the memories it stores — with zero data logging, zero model training, and zero vendor lock-in.

The Agent: A Dynamic, Memory-Aware LangGraph Pipeline

At its core, Theos is a LangGraph state graph with parallel retrieval nodes that executes a multi-step pipeline for every user message:

  1. Embed Query Node — The user's message is embedded using Amazon Titan Embed (amazon.titan-embed-text-v1) on AWS Bedrock. The resulting 1536-dimensional vector is computed once and shared across all downstream nodes to avoid redundant API calls.

  2. Parallel Retrieval — Two nodes execute concurrently:

    • Retrieve Memories Node — Performs a cosine similarity search ($\vec{q} \cdot \vec{m}$) against the ltm_facts table in CockroachDB using Distributed Vector Indexing (VECTOR INDEX). Retrieves the top-$k$ most relevant long-term memories, where $k$ is user-configurable.
    • Retrieve Tools Node — Performs the same vector search against the tool_registry table to find the most relevant tools for the current query. The number of tools retrieved per message is user-configurable (default: 3), directly controlling cost.

  3. Agent Node — The LLM (any open-source model on AWS Bedrock — Mistral, Meta Llama, NVIDIA Nemotron) receives the system prompt, injected memories, conversation history, and resolved tools. Context history is intelligently trimmed based on the model's context limit and the user's configured history percentage.

  4. Dynamic Tool Execution — If the agent calls tools, they're executed and the results loop back to the agent. Tools can include:

    • MCP server tools — dynamically ingested from any MCP-compatible server
    • System tools — save_memories, retrieve_memories, retrieve_additional_tools (the agent can request more tools mid-conversation if the initial set is insufficient), and tavily_search for real-time web search
    • User-defined Python tools — drop a .py file into tools/user_tools/ and it's automatically discovered

Full User Control via the Settings UI

The Theos web interface gives users complete control over:

Setting What It Controls
Model Any of 15+ open-source models on AWS Bedrock
System Prompt The agent's personality, instructions, and behavior
Tone Communication style guidelines
Temperature Response creativity ($0.0$ to $1.0$)
Tools Per Message Number of tools retrieved per query — directly controls cost
History Context % Percentage of the model's context window allocated to conversation history
LTM Limit Number of long-term memories retrieved per query
User Name How the agent addresses the user

The LTM Admin Agent: Conversational Memory Management

Instead of forcing users to write SQL or navigate database tables, Theos provides a specialized LTM Admin Agent in the settings. Users can tell the admin agent their story in natural language, and it will:

  • Re-frame and organize the information into structured facts suitable for agent retrieval
  • Batch insert memories using optimized batch tools (not one-at-a-time)
  • Edit or delete existing memories conversationally

The admin agent is protected by a three-layer safety architecture:

  1. LLM Guard Node — A separate LLM (Mistral Large 675B at temperature 0.0) evaluates every tool call before execution. If the agent attempts to access any table other than ltm_facts (e.g., tool_registry, checkpoints), the guard rejects the call and the agent reflects.
  2. Human-in-the-Loop — All write operations (INSERT, UPDATE, DELETE) require explicit user confirmation via the UI. The system blocks execution and surfaces a confirmation prompt — the user must click "Confirm" before any data changes.
  3. SQL Keyword Detection — Even if the guard approves, the tool execution node independently scans for write keywords and blocks uncommitted writes.

Dynamic MCP Server Integration

Users can add any MCP server — local or cloud-deployed — through the settings UI:

  • Local deployment: Run a Docker container locally, expose it via Cloudflare Tunnel, and paste the HTTPS URL. Theos ingests all tools automatically.
  • AWS deployment: Provide the MCP server config (command, args, env), and Theos deploys it to AWS ECS Fargate as a containerized service behind an Application Load Balancer with CloudFront for HTTPS/TLS — all fully automated, no AWS Console required.

The ingestion engine discovers every tool the MCP server exposes, generates vector embeddings via Amazon Titan, and registers them in the tool_registry table in CockroachDB. From that point on, tools are retrieved dynamically based on semantic relevance to the user's query.

Security Architecture

  • Zero Data Retention: AWS Bedrock never trains on user data. Open-source models run in isolated AWS compute. No LangSmith tracing. No conversation logging.
  • HTTPS/TLS: CloudFront distribution wraps all traffic in TLS — both the Theos app and all MCP server endpoints.
  • Password Protection: The app is protected by a password stored in the .env file. If the user forgets it, they can always check .env.
  • Session Security: Starlette session middleware with a cryptographic secret key.

How We Built It

CockroachDB: The Centralized Brain

Every piece of state in Theos lives in CockroachDB Serverless — it is not merely a database, it is the agent's entire memory infrastructure:

Table Purpose CockroachDB Feature Used
ltm_facts Long-term memories with vector embeddings Distributed Vector Indexing (VECTOR(1536), VECTOR INDEX)
tool_registry All registered tools with semantic embeddings Distributed Vector Indexing for retrieval
app_config User's agent configuration (model, tone, etc.) Standard relational storage
mcp_server_configs MCP server connection configs Relational + JSON columns
conversations Chat thread metadata Relational with timestamps
checkpoints LangGraph conversation state (full checkpointing) Serialized state with optimized queries

We integrated the CockroachDB Cloud Managed MCP Server directly into the LTM Admin Agent — the admin agent connects to cockroachlabs.cloud/mcp using the user's API key and can inspect the cluster, list databases, and query tables through natural language. This is how the memory management agent "sees" the database.

We also used CockroachDB's Distributed Vector Indexing extensively. Both ltm_facts and tool_registry use VECTOR(1536) columns with VECTOR INDEX for cosine similarity search. This means our memory retrieval and tool retrieval are semantic — the agent doesn't just keyword-match, it understands the meaning of the user's query and retrieves the most contextually relevant memories and tools.

AWS: The Infrastructure Backbone

AWS Service How We Use It
Amazon Bedrock All LLM inference (Mistral, Llama, Nemotron) and embeddings (Amazon Titan)
Amazon ECS Fargate Containerized deployment of Theos and MCP servers
Amazon ECR Docker image registry for all containers
AWS CodeBuild Remote Docker builds (no local Docker required)
Elastic Load Balancing (ALB) Shared Application Load Balancer with path-based routing
Amazon CloudFront TLS termination — free HTTPS without custom domains
Amazon CloudWatch Container logging and debugging
Amazon S3 CodeBuild source artifact storage
AWS IAM Least-privilege roles for ECS tasks and CodeBuild

The Stack

  • Backend: Python, FastAPI, LangGraph, LangChain, langchain-aws
  • Database: CockroachDB Serverless (PostgreSQL-compatible, psycopg driver with async connection pools)
  • LLM Provider: AWS Bedrock (Converse API) — open-source models exclusively
  • Embeddings: Amazon Titan Embed Text v1 (1536 dimensions)
  • Deployment: Docker → ECR → ECS Fargate, behind ALB + CloudFront
  • MCP Integration: langchain-mcp-adapters with persistent SSE/streamable-HTTP sessions

Challenges We Ran Into

Efficient MCP Server Deployment. Deploying arbitrary MCP servers as reachable HTTPS endpoints was one of the hardest engineering problems. MCP servers are typically designed for local stdio transport — making them accessible over the network requires a bridge process, a container, networking, health checks, and TLS. We solved this by building a fully automated pipeline: AWS CodeBuild packages the server into a Docker container, pushes it to ECR, and ECS Fargate runs it behind a shared ALB with path-based routing. CloudFront wraps everything in HTTPS. The entire process is a single Python command — no AWS Console needed. AWS's infrastructure-as-code capabilities through boto3 made this pipeline possible, and the reliability of ECS Fargate meant we never had to worry about server management.

Making It Simple for Everyone. We tested Theos with real users across a wide range — adults aged 18–50, teenagers, and kids aged 14–18, including both technical and non-technical people. The feedback was consistent: the setup wizard needed to be dead simple, and the chat interface needed to feel familiar. We iterated heavily on the web wizard (a step-by-step settings flow that handles credentials, configuration, MCP server management, and deployment) and the chat UI to make sure a non-technical user could go from zero to a working agent without touching a terminal.

Centralizing Everything. Early in development, we had configuration in JSON files, memories in a local SQLite database, and tool registries in a separate store. It was fragile and impossible to keep in sync between local and cloud deployments. CockroachDB solved this completely — we moved every piece of state into a single CockroachDB Serverless cluster. Configuration, memories, tools, MCP server configs, conversation checkpoints — all in one place. The PostgreSQL compatibility meant we didn't have to learn a new query language, and the distributed nature meant our cloud deployments and local development could share the same data source seamlessly.

Making It Cost Effective. Using open-source models on Bedrock is already cheaper than closed-source alternatives, but we went further. The dynamic tool retrieval system only loads the tools most relevant to the current query (configurable from 1–15 tools per message), keeping the prompt small. The agent can also request additional tools mid-conversation if the initial set isn't sufficient, rather than loading everything upfront. Memory retrieval is similarly bounded. Combined with configurable context history percentages, users have fine-grained control over exactly how many tokens each message consumes.

Accomplishments That We're Proud Of

  • A truly private AI agent. Zero data retention, zero model training, zero conversation logging. AWS Bedrock's open-source models run in isolated compute — no vendor ever sees user prompts. We audited every data path in the architecture and closed every leak, from debug logs to TLS enforcement.

  • An agent that scales to thousands of tools. Thanks to CockroachDB's Distributed Vector Indexing, Theos can have thousands of tools registered in tool_registry and still retrieve the most relevant ones in milliseconds. The vector index means tool retrieval is $O(\log n)$, not $O(n)$. Most agent frameworks bind a fixed set of tools — Theos retrieves them dynamically per query.

  • The LTM Admin Agent with guardrails. We built an agent to manage another agent's memory — and made it safe. The three-layer guard system (LLM guard, human-in-the-loop, SQL keyword detection) prevents the admin agent from ever touching tables it shouldn't, even if hallucination leads it astray. Users can tell the admin agent their life story in natural language, and it will organize, structure, and store it — all with explicit confirmation before any write.

  • One-command cloud deployment. Clicking the deploy button in the wizard which triggers python deploy_theos_app.py takes the user from zero to a live, HTTPS-secured, globally-accessible agent platform on AWS. No Docker installed locally, no AWS Console clicking, no DNS configuration. CodeBuild, ECR, ECS Fargate, ALB, and CloudFront — all provisioned idempotently.

  • CockroachDB MCP Server integration. The LTM Admin Agent connects directly to the CockroachDB Cloud Managed MCP Server, giving it full read access to the needed tables. This means users can ask the admin agent to inspect the table, describe it, or query facts — all through natural conversation, powered by an official CockroachDB tool.

What We Learned

This hackathon was a deep dive into the CockroachDB and AWS ecosystems, and we came away genuinely impressed.

CockroachDB's Managed MCP Server was a revelation. Being able to connect an AI agent directly to a CockroachDB cluster with a single URL and API key — with built-in audit logging and read-only safety — eliminated an entire class of infrastructure we would have had to build ourselves. The MCP integration felt native, not bolted on.

CockroachDB's Distributed Vector Indexing proved that you don't need a separate vector database for semantic search. Storing embeddings alongside relational data in the same table, with the same transactional guarantees, dramatically simplified our architecture. No sync jobs, no consistency gaps, no extra infrastructure.

AWS Bedrock's open-source model ecosystem showed us that privacy and quality are not mutually exclusive. Mistral Large 675B, Meta Llama 3.3 70B, and NVIDIA Nemotron delivered strong performance while guaranteeing that user data never leaves isolated compute and never trains any model. The Converse API made switching between models a one-line configuration change.

AWS ECS Fargate made containerized deployment painless. We deployed multiple MCP servers and the main application to Fargate without ever managing a server, configuring auto-scaling groups, or worrying about OS patches. Combined with CodeBuild for remote Docker builds, we achieved a fully serverless deployment pipeline.

We would absolutely use CockroachDB and AWS again — they are a powerful combination for building production-grade agentic applications, and we're excited to keep building on this foundation.

What's Next for Theos

  1. Agent Skills System. We're building a skills framework — structured, reusable instruction sets that the agent can activate based on context. Instead of the agent figuring out complex multi-step workflows from scratch each time (burning tokens), skills encode proven workflows that the agent can follow. This makes the agent more cost-efficient and more reliable for repeated tasks.

  2. Parallel MCP Deployment. Currently, MCP servers deploy sequentially. We're moving to parallel deployment — provisioning multiple ECS services, target groups, and listener rules concurrently — to dramatically reduce setup time when users add several servers at once.

  3. Optional Error Reporting. We're adding an opt-in mechanism where, if the agent encounters an error, the user can choose to share the error logs with us to help improve the platform. This is completely optional and off by default — the user must explicitly grant permission, and the shared data is limited to error context only, never conversation content. We've tested extensively across all paths, but with open-source models and dynamic tool resolution, edge cases are inevitable. This feedback loop will help us catch and fix them faster.

  4. Multimodal input payloads. We are extending Theos' capabilities by enabling multimodal input payloads.

  5. Make the MCP server tools ingestion more flexible

Built With

Share this project:

Updates

Submission history