Inspiration
Business decisions rarely come from one voice in a room — they come from finance pushing back on legal, risk flagging what market analysis missed, and a CEO who has to weigh it all and commit. Most "AI decision" tools collapse that into a single model giving a single answer, with no memory of what was decided last time and no visible disagreement along the way.
We wanted to see if a fixed panel of specialized agents — CEO, Finance, Legal, Market Analyst, Risk Analyst, and Project Manager — could actually argue with each other, on the record, and land on a verdict a human could audit afterward. And we wanted the system to remember, so a follow-up question six weeks later doesn't start from zero.
What it does
You ask a business question — for example, "Should we launch an AI healthcare product in India?" — and:
- The system recalls any relevant past decisions from persistent memory and surfaces them to every agent before the meeting starts.
- Six agents take turns in a fixed meeting order, each seeing prior turns and recalled memory as they form a position.
- If two agents' positions conflict, they get one rebuttal round before the meeting concludes.
- The CEO agent synthesizes everything into a structured verdict: recommendation, risk score, cost estimate, timeline, confidence, and references to any memory it relied on.
- The verdict is persisted so future related questions recall it automatically.
The whole pipeline runs autonomously end to end — no human has to step in between the question and the verdict.
How we built it
- Orchestration sequences the six agents in a fixed meeting order, detects when two positions conflict, and runs a single rebuttal round before handing off to synthesis.
- Memory is a SQLite-backed decision store with embedding-based recall, so related questions pull in prior verdicts instead of re-litigating them from scratch.
- Agents are built as distinct personas with their own prompt construction, so each brings a genuinely different lens (leverage risk vs. regulatory timeline vs. market upside) rather than the same model restating itself six ways.
- Clients are wrapped behind a
QwenClientabstraction (DashScope + a fake implementation for tests), plus an Alibaba Cloud Log client so every verdict has a durable, inspectable trail. - The API is FastAPI, streaming the live meeting to a static frontend SPA over server-sent events, so you watch agents disagree in real time rather than waiting on a spinner.
- Deployment is Alibaba Cloud ECS via Docker, with decision verdicts written to Alibaba Cloud Log Service as the proof-of-deployment artifact.
Challenges we ran into
- Making disagreement real, not decorative. It's easy to have agents nominally represent different roles while converging instantly on the same answer. Getting genuine, useful conflict — and a rebuttal round that actually resolves rather than loops — took real tuning of how much context each agent sees and when.
- Testing a multi-agent pipeline without live LLM calls. All tests
run against a
FakeQwenClientwith no live network calls in CI, which meant carefully designing fakes that vary their stance per agent — otherwise the conflict-detection and rebuttal code paths look covered but never actually execute. - Memory recall precision. Surfacing genuinely relevant past decisions (not just similar-sounding ones) to every agent, without flooding the prompt, took iteration on the embedding recall step.
- Scoping a 48-hour build. We made deliberate cuts — fewer agents, no RAG over documents, no knowledge-graph visualization, a single orchestration approach — documented as ADRs so the reasoning isn't lost, just deferred.
What we learned
Multi-agent "debate" is only as good as the orchestration around it — the interesting engineering isn't the agents' prompts, it's the conflict-detection and rebuttal logic that decides when they need to argue and when to stop. We also came away convinced that persistent, recallable memory changes the character of these systems: a decision tool that remembers what it told you last time behaves very differently from one that starts fresh every call.
What's next
- Expanding beyond a fixed six-agent panel to configurable panels per domain.
- RAG over uploaded documents (financial statements, contracts) so agents can ground positions in a company's actual data.
- A knowledge-graph view of how past decisions relate to each other, beyond flat embedding recall.
Log in or sign up for Devpost to join the conversation.