Inspiration

Business decisions rarely come from one voice in a room — they come from finance pushing back on legal, risk flagging what market analysis missed, and a CEO who has to weigh it all and commit. Most "AI decision" tools collapse that into a single model giving a single answer, with no memory of what was decided last time and no visible disagreement along the way.

We wanted to see if a fixed panel of specialized agents — CEO, Finance, Legal, Market Analyst, Risk Analyst, and Project Manager — could actually argue with each other, on the record, and land on a verdict a human could audit afterward. And we wanted the system to remember, so a follow-up question six weeks later doesn't start from zero.

What it does

You ask a business question — for example, "Should we launch an AI healthcare product in India?" — and:

  1. The system recalls any relevant past decisions from persistent memory and surfaces them to every agent before the meeting starts.
  2. Six agents take turns in a fixed meeting order, each seeing prior turns and recalled memory as they form a position.
  3. If two agents' positions conflict, they get one rebuttal round before the meeting concludes.
  4. The CEO agent synthesizes everything into a structured verdict: recommendation, risk score, cost estimate, timeline, confidence, and references to any memory it relied on.
  5. The verdict is persisted so future related questions recall it automatically.

The whole pipeline runs autonomously end to end — no human has to step in between the question and the verdict.

How we built it

  • Orchestration sequences the six agents in a fixed meeting order, detects when two positions conflict, and runs a single rebuttal round before handing off to synthesis.
  • Memory is a SQLite-backed decision store with embedding-based recall, so related questions pull in prior verdicts instead of re-litigating them from scratch.
  • Agents are built as distinct personas with their own prompt construction, so each brings a genuinely different lens (leverage risk vs. regulatory timeline vs. market upside) rather than the same model restating itself six ways.
  • Clients are wrapped behind a QwenClient abstraction (DashScope + a fake implementation for tests), plus an Alibaba Cloud Log client so every verdict has a durable, inspectable trail.
  • The API is FastAPI, streaming the live meeting to a static frontend SPA over server-sent events, so you watch agents disagree in real time rather than waiting on a spinner.
  • Deployment is Alibaba Cloud ECS via Docker, with decision verdicts written to Alibaba Cloud Log Service as the proof-of-deployment artifact.

Challenges we ran into

  • Making disagreement real, not decorative. It's easy to have agents nominally represent different roles while converging instantly on the same answer. Getting genuine, useful conflict — and a rebuttal round that actually resolves rather than loops — took real tuning of how much context each agent sees and when.
  • Testing a multi-agent pipeline without live LLM calls. All tests run against a FakeQwenClient with no live network calls in CI, which meant carefully designing fakes that vary their stance per agent — otherwise the conflict-detection and rebuttal code paths look covered but never actually execute.
  • Memory recall precision. Surfacing genuinely relevant past decisions (not just similar-sounding ones) to every agent, without flooding the prompt, took iteration on the embedding recall step.
  • Scoping a 48-hour build. We made deliberate cuts — fewer agents, no RAG over documents, no knowledge-graph visualization, a single orchestration approach — documented as ADRs so the reasoning isn't lost, just deferred.

What we learned

Multi-agent "debate" is only as good as the orchestration around it — the interesting engineering isn't the agents' prompts, it's the conflict-detection and rebuttal logic that decides when they need to argue and when to stop. We also came away convinced that persistent, recallable memory changes the character of these systems: a decision tool that remembers what it told you last time behaves very differently from one that starts fresh every call.

What's next

  • Expanding beyond a fixed six-agent panel to configurable panels per domain.
  • RAG over uploaded documents (financial statements, contracts) so agents can ground positions in a company's actual data.
  • A knowledge-graph view of how past decisions relate to each other, beyond flat embedding recall.

Built With

Share this project:

Updates