Building CrewFactory: My Experience Assembling a Multi-Agent System for the Qwen Cloud Hackathon

CrewFactory was born as a response to a concrete challenge: the Qwen Cloud hackathon, Agent Society track. The brief asked for a system where multiple agents with different capabilities would collaborate through task division, dialogue, and negotiation to solve complex problems — and to demonstrate, with data, that this teamwork outperformed a single agent solving everything on its own.

That phrase — "measurable efficiency gain over a single agent" — ended up being the compass for the entire project. Every architectural decision, from how agents delegate tasks to how they resolve a disagreement, had to answer to that question.

I started with the bare minimum: a real-time streaming chat connected to an agent runtime, with authentication and sessions. The boring-but-necessary foundation to iterate on. Weeks later, that foundation had grown into a full multi-agent orchestration platform: group chat channels, teams with negotiation and arbitration, a permissions engine with sandboxing, complete observability of every session, and a layered prompt system that gives each agent its own identity and role within the team.

Task division: from a single prompt to a team with roles

The first thing I had to solve for the track was literal: how does a group of agents decide who does what? The first version was naive — one agent received the whole task and worked through it alone. That's not collaboration, it's just delegation in disguise.

The real solution came with task decomposition: a "planner" agent receives the goal, breaks it into concrete subtasks, and those subtasks get assigned to specialized agents based on their declared role (backend, frontend, testing, deploy, etc.). Each agent executes its part in its own isolated session, with its own history and its own context — no shared memory that could contaminate another agent's work. When an agent needs another's output, it waits for it explicitly and resumes execution as soon as that result arrives, instead of constantly polling.

The detail that took the longest to get right: at first, planning went through a full agent session (slow, expensive in tokens). I ended up replacing it with a direct, deterministic call that just asks "split this into tasks" without the full weight of a complete conversation — same result, a fraction of the cost and latency.

Dialogue and negotiation: when agents don't agree

This is the part most connected to the spirit of the track. It's not enough for agents to split up the work; they need to be able to argue when there's disagreement — for example, two agents proposing different technical solutions to the same problem.

For this I built "negotiation teams": a mode where several agents debate a proposal over rounds, each one votes (in favor, against, or with reservations), and a consensus engine evaluates whether there's enough quorum to close the round or whether discussion needs to continue. If there's no agreement after several rounds, an arbiter agent breaks the tie.

I had to put hard limits on this quickly: without a cap on rounds and on delegation-chain depth, a disagreement between agents can easily turn into an infinite loop where they just keep citing each other without reaching any conclusion. That was one of the most expensive bugs to diagnose in the whole project — it didn't break anything visibly, the system just kept "working" without making progress. The fix was an explicit circuit breaker: a configurable maximum chain depth, and anti-chatter rules so an agent doesn't respond just out of courtesy when it has nothing technical to contribute.

Measuring efficiency: the requirement that's almost always skipped

The part of the track that's easiest to skip is "demonstrate measurable improvement." It's tempting to build the multi-agent system and just assume it's better because it sounds more sophisticated. To avoid that, I built an experiment lab: it runs the same task in two variants — a single agent (baseline) and a team with negotiation — and compares real metrics: tokens consumed, total time, number of tool calls, error rate.

That analytics dashboard ended up being indispensable, not just for the hackathon, but for my own design judgment: every time I added a coordination layer between agents, I could immediately see whether that extra complexity translated into a better result or just more overhead. Several times the answer was "more overhead," and I had to simplify instead of adding sophistication.

Observability: seeing what the agents are doing matters as much as them working

Once you have several agents working in parallel, the question that becomes urgent fastest is: what is each one doing right now? I built a kanban-style board with each session's state (idle, working, finished), real-time activity indicators in the sidebar, and a vertical timeline inside each chat showing every thought, every tool call, and every response with its duration.

For teams specifically, I built a "swimlanes" view — parallel lanes showing, turn by turn, which agent said what and when, visually connected like a sequence diagram. Without this, reading the log of a debate between four agents is unreadable; with it, you can tell at a glance who convinced whom.

What I'm taking away

The hackathon's requirement — real collaboration, not just delegation, with measurable improvement to back it up — turned out to be the best design filter I could have had. Any time a feature didn't help divide tasks better, resolve a disagreement, or demonstrate a real gain, I set it aside.

If you're building something similar, my advice is that: don't design the complete multi-agent system from day one. Build the minimum coordination needed to resolve a real disagreement, measure whether it actually improves anything over a single agent, and let that measurement — not intuition — tell you what to build next.

Built With

• bun
• qwen