Orchestra

Inspiration

Multi-agent AI systems are supposed to be more reliable than single agents — more eyes, better quality. But in practice, the opposite happens: errors propagate and amplify. Agent A makes a mistake, Agent B doesn't challenge it but builds on it, and the final output is worse than if you'd used just one agent. This is reviewer sycophancy, and it's the unsolved problem in multi-agent collaboration.

What it does

Orchestra is a visual multi-agent workflow platform that embeds quality control directly into the execution engine. Users build agent topologies on a drag-and-drop canvas — Producer, Reviewers, Adversarial Agent, Convergence gates — and watch them collaborate in real time. The core innovation is Adversarial Convergence: an adversarial red-team agent that challenges reviewer assessments when it finds issues they missed, reducing sycophancy by 68% and catching 24% more bugs compared to standard multi-agent review.

How we built it

Zero-dependency vanilla HTML/CSS/JS — no frameworks, no build tools. The orchestration engine implements Kahn's algorithm for DAG topological sorting, with async/await-based parallel node execution. Each agent node calls Qwen Cloud API (DashScope) via streaming SSE for real-time output. The adversarial layer was the key architectural decision: instead of simple majority vote or retry, we built a dedicated challenge agent that examines both reviewer outputs AND the original content, injecting feedback upstream when it finds missed flaws.

Challenges we ran into

The hardest problem was getting agent nodes to actually see each other's outputs. The initial implementation ran each agent in isolation — reviewers had no access to the producer's output, so they couldn't review anything. Fixing the upstream data injection was critical to making the convergence loop work. Another challenge was the DashScope API integration: CORS, streaming SSE parsing, and handling the reasoning_content field that qwen3.7-plus returns alongside regular content.

Accomplishments that we're proud of

The full adversarial convergence loop works end-to-end: Producer generates content → two Reviewers score independently → Adversarial Agent catches what they missed → feedback loops back to Producer. In our benchmark (5 test cases, 23 known flaws), adversarial convergence caught 21/23 bugs vs 17/23 for standard convergence, while reducing sycophancy from 24.6% to 7.8%. This is a real, measurable improvement — not just a demo trick.

What we learned

Deterministic convergence is hard. Random retry doesn't work — you need structured feedback injection with diminishing-returns detection. The adversarial layer was unexpected: we thought two reviewers would be enough, but they consistently gave each other high scores. The adversarial agent was the missing piece that made the system actually reliable. Also learned that vanilla JS can build surprisingly complex interactive applications without any framework.

What's next for Orchestra

Native macOS version with SwiftUI Canvas (already in development). Plugin system for custom agent types. Team collaboration — multiple users editing the same workflow. And expanding beyond text review: code review, image analysis, data validation — anywhere multi-agent quality control matters.

Built With

• ai
• alibabacloud
• css3
• dashscope
• html5
• javascript
• multi-agent
• qwen