Agent Society Observatory

Inspiration

Most agent demos show agents talking. Agent Society Observatory asks a sharper question: on the same PRD, did a structured agent society catch a requirement or risk that a solo agent missed, and was the extra token and latency cost worth it?

What it does

The app compares a solo-agent baseline with a constitutional-review agent society. It shows the same PRD moving through a direct solo draft, a specialist objection, a PM revision, and a reviewer gate. The dashboard keeps the evidence attached: score deltas, token usage, latency, objection alignment, claim boundaries, and reusable prd_rubric_demo rows.

How we built it

The experiment runner uses Qwen Cloud through the OpenAI-compatible DashScope endpoint. It records paired protocol runs, exports scorecards and arena-safe comparison data, and feeds a React/Vite evidence console. The backend proof service runs on Alibaba Cloud Function Compute and calls Qwen Cloud server-side at /api/proof/qwen-smoke.

Challenges we ran into

The main challenge was avoiding overclaiming. The data is useful selected-task protocol evidence, not a universal model ranking and not hidden-oracle benchmark evidence. The UI therefore keeps Qwen provenance, Alibaba proof status, and claim boundaries visible instead of hiding them in raw files.

Accomplishments that we're proud of

Public evidence console with animated agent-society replay.
Qwen Cloud model-evidence rows and Qwen-ready story/proof surfaces.
Alibaba Function Compute proof endpoint with live Qwen smoke calls.
Public repo with architecture diagram, deployment code, and generated submission packet.

What we learned

Agent societies are most compelling when their disagreement is auditable. A better answer is not enough; the product needs to show what objection was raised, how it changed the output, and what cost it added.

What's next for Agent Society Observatory

Next steps are broader task packs, richer role protocols, and a cleaner export path for other AI Regime Arena provider-comparison datasets while preserving the same claim-boundary discipline.