Decision Council

Decision Council is an observable multi-agent deliberation app that assembles specialized AI advisors around a business decision, debate tradeoffs, and produce a traced, auditable recommendation.

Comment

Inspiration

I'll be honest with you. I have no business building this.

No computer science degree. No software industry experience. What I have is a C programming course from college, a Java class, something called machine language that I mostly blocked out, and about twenty years of tinkering with HTML because I thought it was fun. That's it. That's the résumé.

But I kept watching this AI wave building and something in me just wouldn't sit still. This felt like one of those moments — the kind that only comes around a few times in a lifetime — where the ground shifts under everything and the people who show up early get to help decide what gets built. I didn't want to watch that from the sidelines.

So I started building. Badly at first. Then less badly.

For weeks I was doing what I now affectionately call "herding squirrels" — bouncing between two or three LLM windows, copy-pasting outputs, trying to orchestrate something resembling useful AI work. And somewhere in that chaos I noticed something I couldn't un-notice. These things will agree with almost anything. Push back a little, reframe the question, come at it sideways — and more often than not the model bends. Not because it's wrong exactly, but because somewhere deep in its training is this overwhelming need to be helpful, which in practice sometimes looks a lot like being agreeable.

I started calling it the "yes problem."

And the more I sat with it, the more I realized: if you're a business owner thinking about handing real decisions to an AI agent — booking appointments, processing orders, responding to your customers — trust is the whole game. Not capability. Trust. Nobody mainstreams a worker they don't trust. Human or otherwise.

That's when the idea hit me. What if instead of me bouncing between three browser windows trying to triangulate honest answers, I put the agents in a room together?

What if one agent had to make the case, another had to challenge it, another had to think about risk, another had to protect customer trust, and the whole process was traced so we could inspect what actually happened?

That became Decision Council..

What it does

Decision Council is a Gemini-powered agent system that helps business owners evaluate high-stakes decisions with more structure, dissent, and observability.

A user enters a business decision, chooses a council type, and receives a recommendation built from multiple advisor perspectives. The council runs independent advisor rounds, a challenge round, final synthesis, quality evaluation, and applied improvement directives.

The demo includes three preset councils:

AI Adoption Council for deciding whether and how a business should use AI. Risk Review Council for reviewing safety, compliance, security, and operational risk. Launch Review Council for evaluating whether a product, feature, or initiative is ready to go live.

Decision Council also includes a Google ADK entrypoint that exposes the council as a callable agent tool. A parent ADK agent can invoke the council and optionally inject a Gemini-powered subject-matter expert, such as a roofing operations expert, healthcare compliance advisor, or payments risk reviewer.

Every run is traced in Arize Phoenix, including the council run, advisor rounds, synthesis, and evaluation span.

How we built it

The public demo is a code-owned Python/FastAPI web agent deployed on Google Kubernetes Engine Autopilot. Gemini on Vertex AI powers the advisor reasoning, challenge round, synthesis, and improvement directives.

We used Google Cloud for the core runtime and deployment path: Vertex AI, GKE Autopilot, Artifact Registry, Cloud Build, IAM, Workload Identity, and Secret Manager. We also validated Cloud Run and Firebase Hosting during deployment, but the final public demo uses GKE because inherited organization policy blocked public Cloud Run invoker access.

For observability, we instrumented the agent with Arize Phoenix, OpenInference, and OpenTelemetry. Phoenix receives traces for each council run so the decision process can be inspected after the fact.

We also added a Phoenix MCP configuration path and smoke test so the project can support runtime introspection over Phoenix operational data.

Challenges we ran into

The hardest part was not just getting an answer from Gemini. The hard part was making the answer trustworthy, inspectable, and less generic.

We had to move from a single-response chatbot pattern into a real deliberation pattern: independent advisors, dissent, challenge, synthesis, evaluation, and improvement directives.

Deployment also came with real-world friction. Cloud Run worked, but public unauthenticated access was blocked by inherited organization policy. Firebase Hosting successfully deployed, but its Cloud Run rewrite still hit the same invoker restriction. We eventually deployed the public demo through GKE Autopilot with a LoadBalancer service.

We also had to work through GKE Workload Identity so the running pod could call Gemini on Vertex AI securely without embedding credentials in the app.

Accomplishments that we're proud of

We built a working public agent demo that runs on Google Cloud, uses Gemini for real decision work, traces the decision process to Arize Phoenix, and exposes a Google ADK entrypoint.

The part I’m most proud of is the self-improvement loop. The system does not just produce a recommendation. It evaluates the quality of the verdict, generates improvement directives, and applies those directives to make the next answer more specific and useful.

I’m also proud that the demo can inject a custom expert into the council without using another LLM. The expert becomes another Gemini-powered advisor role inside the same traced council runtime.

And personally, I’m proud that this went from “I have no business building this” to a deployed public system with GKE, Vertex AI, Phoenix traces, MCP configuration, and ADK compatibility.

What we learned

We learned that agent quality is not just about model capability. It is about structure.

A single AI answer can sound confident even when it is vague. A council pattern creates friction: different roles, different incentives, dissent, safeguards, and evaluation. That friction makes the output better.

We also learned that observability matters early. Tracing is not something to bolt on later. When agent systems are making recommendations, you need to see the path they took to get there.

Finally, we learned that “agent” should not mean “magic box.” The more useful pattern is a clear runtime with tools, roles, traces, and improvement loops

What's next for Decision Council

Next, we want to make Decision Council more adaptive and domain-aware.

The ADK entrypoint opens the door for other Google agent workflows to call the council as a tool and inject subject-matter experts when needed. A mock trial could add a plaintiff attorney, defense attorney, judge, and expert witness. A boardroom review could add finance, legal, customer trust, and operations. A product launch could add market skeptic, security, revenue, and support burden reviewers.

We also want to deepen the Phoenix MCP loop so the council can query its own past traces, identify recurring weaknesses, and improve its prompts and evaluation patterns over time.

Longer term, Decision Council could become a trust layer for AI-assisted judgment: not just “what does the AI recommend?” but “how was the recommendation debated, challenged, evaluated, and improved?”

Built With

  • arize-phoenix
  • artifact-registry
  • cloud-build
  • docker
  • fastapi
  • gemini-on-vertex-ai
  • gke
  • google-cloud
  • google-iam-/-workload-identity
  • google-kubernetes-engine-autopilot
  • html/css/javascript
  • kubernetes
  • openinference-/-opentelemetry-tracing
  • phoenix-mcp
  • python
Share this project:

Updates