BI Guardian Operator

An autonomous Power BI engineering agent that inspects projects locally, runs deterministic audit tools, reasons with Gemini, and produces an evidence-backed release-readiness handoff.

Comment

Inspiration

Power BI engineering teams already have analyzers, rules, warnings, and documentation tools. The harder problem is deciding what all that information means for the next engineering step.

Is the project actually ready for production? Is the technical handoff complete? Can a governance claim really be verified from the available project evidence?

BI Guardian Operator started from a simple idea:

The engines know. The agent coordinates. The AI interprets. Evidence rules.

Instead of asking an LLM to inspect a Power BI project and simply generate an answer, I wanted to build an agent that could coordinate real engineering work while keeping deterministic tools and evidence authoritative.

What it does

BI Guardian Operator accepts a real Power BI PBIP or TMDL project and inspects its metadata locally in the browser.

The user gives Operator an engineering goal, and the agent coordinates bounded capabilities until the mission has enough evidence to produce a justified outcome.

The demo shows three missions running through the same governed Operator runtime:

Review Before Production

Operator coordinates deterministic audit capabilities, gathers findings and evidence, and produces a release-readiness outcome such as Ready or Ready with Conditions.

Findings remain individually traceable to their deterministic rule, severity, classification, evidence, and score impact.

Technical Handoff

The same Operator runtime is given a different goal.

It gathers project inventory, maintainability signals, findings, and engineering work items, then produces durable artifacts including a structured technical handoff and a semantic-model diagram derived from project evidence.

Governance Review

Operator also evaluates governance claims without manufacturing certainty.

When evidence supports a claim, it can be verified.

When the project cannot establish the answer, the result can remain UNKNOWN or NOT OBSERVABLE.

Missing evidence is never silently converted into a pass.

How we built it

BI Guardian Operator deliberately separates reasoning, execution, and engineering truth.

  • Deterministic Power BI engines inspect project metadata and establish observable engineering facts.
  • Google Agent Development Kit (ADK) coordinates mission execution and bounded capabilities.
  • Gemini on Vertex AI interprets mission state and selects among allowed next actions.
  • Mission-specific outcome contracts define what evidence is required before a goal can be considered complete.
  • Evidence and execution traces preserve what was observed and why an outcome was reached.
  • Google Cloud Run hosts the Operator backend.
  • Firestore stores sanitized operational mission state.
  • Raw PBIP/TMDL project contents remain local; only sanitized mission evidence is allowed to reach cloud orchestration.

A mission follows a loop similar to:

Goal → bounded action → deterministic execution → evidence → mission-state update → re-evaluation → completion check

This allows the same Operator runtime to support multiple engineering missions without hard-coding a completely different agent for each one.

Prior-work boundary

BI Guardian Operator builds on a pre-existing deterministic Power BI analysis core.

For the hackathon, that core remained a defined boundary while the new agentic Operator layer was built around it: mission orchestration, bounded capability coordination, evidence-governed execution, mission outcome contracts, operational state, Google ADK/Gemini integration, Cloud Run deployment, Firestore persistence, and the multi-mission experience demonstrated in the submission.

The separation is intentional:

the deterministic engines provide specialized BI knowledge; Operator governs and coordinates them into complete engineering missions.

Challenges we ran into

Keeping AI reasoning separate from project truth

One of the hardest architectural problems was preventing the language model from becoming both the reasoner and the source of truth.

Gemini can interpret mission state and choose an allowed action, but deterministic engineering engines remain authoritative for observable Power BI facts.

Deciding what “done” means

A successful tool call does not necessarily mean the engineering goal has been achieved.

We introduced explicit outcome contracts so that mission completion depends on required evidence instead of model confidence.

Representing uncertainty honestly

Some governance questions cannot be answered from PBIP/TMDL metadata.

Instead of converting missing information into false, failed, zero, or passed, Operator preserves states such as UNKNOWN and NOT OBSERVABLE.

Preserving privacy

We wanted to benefit from Gemini and Google Cloud without unnecessarily uploading an entire Power BI project.

This led to the local-analysis and sanitized-evidence boundary used by the current architecture.

Accomplishments that we're proud of

The biggest accomplishment is that BI Guardian Operator is not just a chatbot sitting in front of an analyzer.

It can coordinate a real multi-step engineering workflow:

understand the goal → choose an allowed capability → execute deterministic analysis → gather evidence → re-evaluate the mission → determine what is still missing → continue or close → produce a durable outcome

We are also proud that the same governed runtime successfully supports three materially different missions rather than relying on three separate hard-coded workflows.

Another important result is the separation between:

mission completion and engineering outcome.

A mission can complete correctly while still reporting Ready with Conditions, unresolved findings, or unknown governance states.

Finally, the production system is deployed on Google Cloud Run, with Google ADK, Gemini on Vertex AI, and Firestore forming the cloud agent stack while preserving the local privacy boundary around the original Power BI project.

What we learned

The strongest lesson was that reliable agentic systems do not require the LLM to know everything.

A more robust division of responsibility emerged:

  • deterministic systems establish facts,
  • the agent coordinates work,
  • the model interprets context and chooses among allowed actions,
  • evidence determines what can legitimately be claimed.

We also learned that completion itself should be treated as an engineering contract.

For multi-step agents, knowing whether the system has sufficient evidence to stop can be just as important as deciding what action to perform next.

And finally, uncertainty is not necessarily failure.

In governed engineering systems, being able to say:

“This cannot be established from the available evidence.”

can be more trustworthy than producing a confident answer.

What's next for BI Guardian Operator

The current version demonstrates the architecture through three Power BI engineering missions.

The next step is to expand the capability library and support additional BI engineering goals while preserving the same execution principles:

bounded capabilities, deterministic domain tools, explicit evidence authority, traceable mission state, and justified completion.

The longer-term goal is for BI Guardian Operator to become an engineering control plane capable of coordinating increasingly complex Power BI workflows without sacrificing traceability or governance.

The larger question behind the project remains:

Can an AI agent perform real engineering work without being allowed to manufacture certainty?

BI Guardian Operator is our first answer.

Built With

Share this project:

Updates

posted an update —

I designed and built BI Guardian Operator as an individual project, including the agentic architecture, mission orchestration, Google ADK and Gemini integration, evidence-governed execution, Cloud Run deployment, Firestore persistence, Power BI engineering workflows, product experience, and technical documentation.

Log in or sign up for Devpost to join the conversation.