analyst copilot

Inspiration

Most AI-for-BI demos look magical until they meet production. The model invents a metric, invents a join, invents a number—and the UI still smiles. That failure mode inspired Analyst Copilot.

We did not want another chatbot that “talks about dashboards.” We wanted an agent grounded in an enterprise operational Ontology: a system where business meaning is first-class, physical plumbing stays private, and every answer or action can be verified.

The spark came from a simple inversion:

Don’t force the LLM to understand the warehouse. Make the business world itself tool-shaped.

Inspired by public operational-ontology ideas (especially Ontology + Ontology MCP style tool exposure), we set out to build an agent that reads the tools a current app and current user are allowed to use, then autonomously investigates, verifies, continues reasoning, and—when policy allows—acts.

What it does

Analyst Copilot is a complete ontology-native agent for governed analytics and controlled operations.

For the end user, it feels like an agent that:

• understands business questions in natural language,
• queries real business objects,
• calls governed business calculations,
• inspects evidence instead of vibes,
• continues drilling when the first answer is incomplete,
• and can execute real business actions under policy.

Under the hood, the system keeps a strict split:

• public business semantics for the model,
• private physical bindings for the runtime,
• deterministic compilation and gates for execution,
• immutable evidence / receipts for accountability.

So the agent can think freely, but it cannot invent tables, skip authorization, or quietly rewrite reality.

In short:

$$ \text{User Goal} \rightarrow \text{Authorized Ontology Tools} \rightarrow \text{Evidence / ActionReceipt} \rightarrow \text{Answer or Next Move} $$

How we built it

We built Analyst Copilot as a Python backend-first system with two complementary engines:

1. a model-driven QueryLoop for thinking and tool selection, and
2. a deterministic Ontology Runtime / control plane for binding, compile, permission, execution, and audit.

Ontology as the single world model

Objects, properties, links, values, query functions, action types, and policies all live in one Enterprise Ontology. Facts stay in source systems such as Tableau Cloud; the ontology stores stable business identity and private mappings rather than rebuilding a warehouse.

Dynamic tool projection

Before the LLM sees anything, the system intersects:

• enterprise ontology version,
• application scope,
• principal permissions / RLS.

A tool projector then emits only the authorized business tool schemas. The model never browses the whole enterprise—only the slice it may use.

Agent loop without workflow theater

The LLM reads tool names, descriptions, and JSON Schema; chooses calls; fills business parameters; reads evidence; and decides the next step. There is no fixed slot-filler and no global analysis DAG pretending every question is a pipeline.

Runtime as the citadel

Tool calls resolve private bindings, compile privately, pass safety/quality/action gates, execute, and freeze evidence or receipts. Skills packages supply analytical playbooks, while required-tool gates stop “pretend completion.”

Real proof path

Deterministic tests cover compile/bind/safety logic. Capability claims are proven with live tests against real Tableau Cloud and a real LLM (DeepSeek), not stubs posing as competence.

Challenges we ran into

Public meaning vs private plumbing

If the model sees tables and SQL, it starts planning like a warehouse engineer. If it sees too little, it cannot act. Drawing that boundary cleanly—and keeping it leak-free—was one of the hardest design problems.

Autonomy without a brittle workflow

Analysis is branching and evidence-driven. We needed strong runtime control without freezing strategy into a fake flowchart. The answer was: model owns the next step; server owns the gates.

Ambiguity without over-ceremony

Not every noun deserves a selection ritual. Only true ambiguity—multiple currently legitimate candidates—should create a continuation/commitment. Too much ceremony slows the agent; too little creates silent wrong grounding.

Actions that are real but not reckless

“Can act” is dangerous. Write-backs had to become first-class ActionTypes with policy tiers, receipts, and auditability. Otherwise the agent becomes a liability with good prose.

Refusing mock-as-proof

Live verification is expensive, messy, and humbling. It also exposes the only failures that matter: dialect edges, degraded profiles, context overflow, permission boundaries, and the gap between plausible text and accountable results.

Accomplishments that we're proud of

• A coherent ontology-native architecture where the model reasons over business tools, not warehouse debris.
• A clean separation of public semantics and private bindings.
• A production-shaped loop: QueryLoop for autonomy, Ontology Runtime for determinism.
• Evidence-backed answers and policy-gated actions instead of “trust me” narratives.
• A verification culture that treats live Tableau + live LLM tests as the real proof of capability.
• Turning a vague “AI BI assistant” dream into a sharper product thesis:

Make the business world tool-shaped for the model, and keep the physical world server-shaped for the truth.

What we learned

We learned that agent quality is mostly an ownership problem:

• the model should own natural-language understanding and next-step strategy;
• the ontology should own business meaning;
• the runtime should own physical truth, compilation, authorization, and side effects;
• evidence should own accountability.

We also learned that menus are discovery surfaces, not authority; that autonomy without gates is elegant hallucination; and that production-grade agents are built less by clever prompting than by uncompromising boundaries.

A useful mental model became:

$$ \text{Trust}(a)=\mathbf{1}{\text{grounded}}(a)\cdot\mathbf{1}{\text{authorized}}(a)\cdot\mathbf{1}_{\text{evidence-backed}}(a) $$

If any term is zero, the system should refuse or ask—not invent.

What's next for analyst copilot

Next we want to push the ontology-native complete agent further into daily enterprise use:

• deepen ActionType write-back coverage with richer policy, confirmation, and compensation paths;
• expand QueryFunctions for cross-source business calculations;
• strengthen application-scoped tool projection and principal-aware menus;
• harden long-running investigation memory across multi-turn drills;
• grow live case packs from real governed failures;
• and ship a polished user-facing chat/workbench that makes evidence, continuations, and action receipts first-class in the UI.

The destination is clear: an agent that does not just answer questions about the business, but can operate inside the business—carefully, verifiably, and under governance.

Built With

• agents
• api
• deepseek
• duckdb
• fastapi
• httpx
• jwt
• llm
• mcp
• openai
• postgresql
• pydantic
• pytest
• python
• sqlglot
• sse
• tableau
• uvicorn