Escalation Room Society

Seven Qwen agents investigate a family's complaint about a Medicaid therapist listing, verify it using real sources and a phone call, then publish only evidence they can defend.

Comment

Inspiration

Finding a therapist through Medicaid can be frustrating. Families often spend hours calling providers, only to find disconnected numbers, outdated information, or practices that stopped accepting Medicaid long ago. Every dead end delays someone who may need help urgently.

We realized this isn't just a search problem. It's a maintenance problem. Directories become outdated because keeping them accurate is repetitive, ongoing work. We wanted to explore whether a society of AI agents could continuously investigate reports from families, gather evidence, and make the directory more trustworthy over time.

What it does

Escalation Room Society turns a family's report into an investigation.

In our demo, a family reports that a therapist's phone number no longer works. Instead of relying on a single source, specialized AI agents independently gather evidence from public profiles, official records, previous verification history, and even a real phone verification. As new evidence arrives, the provider page updates in real time, confidence increases, and every change is backed by a clear verification history.

Not every decision should be automated. When the evidence isn't strong enough for a high-impact field, the system deliberately stops and flags it for human review instead of guessing.

The result is a directory that becomes more trustworthy with every resolved report.

How we built it

We built the project as a society of seven Qwen-powered agents, each responsible for a different part of the investigation. Rather than debating endlessly, each agent contributes evidence that others can build upon until the system reaches a decision.

To make the investigation easy to understand, we designed the experience around a simple six-step journey instead of raw agent logs. Viewers can watch evidence arrive, confidence increase, and the provider page improve as the investigation unfolds.

The entire system can run deterministically for demonstrations, while also supporting live verification against real public sources.

Challenges we ran into

The biggest challenge wasn't building multiple agents. It was making their collaboration understandable.

Early versions looked like agents talking to each other, but that wasn't compelling. We redesigned the experience so viewers follow the investigation itself instead of the conversations behind it.

Another challenge was deciding where AI should stop. Updating a phone number is relatively low risk, but incorrectly changing whether a provider accepts Medicaid could remove a real option for a family. That led us to design the system around evidence and confidence instead of blind automation.

Accomplishments that we're proud of

  • Built a multi-agent investigation that gathers evidence from real sources instead of relying on a single model.
  • Included a real phone verification as part of the investigation, showing an AI agent taking meaningful action beyond passive lookups.
  • Designed a confidence-based publishing workflow that updates low-risk information automatically while reserving sensitive decisions for human review.
  • Created a directory that permanently improves as each report is resolved.

What we learned

Building multiple agents wasn't the hard part. Building trust was.

We learned that agents become much more useful when they produce evidence instead of opinions. We also learned that knowing when not to automate is just as important as automation itself. Sometimes the best decision an AI system can make is to ask for human judgment.

Most importantly, we found that showing a shared artifact improving over time is far more compelling than showing agents simply talking to one another.

What's next

Our goal is to expand beyond a single demonstration into a continuously maintained directory that families can genuinely rely on. We plan to verify more providers, incorporate additional evidence sources, and continue improving how AI and humans work together to keep information accurate.

If you'd like to learn more about the thinking behind the project, including the design journey, tradeoffs, and how this project connects with our companion hackathon submission, we've documented the full story on Medium:

https://medium.com/@michiyamamoto/escalation-room-when-ai-investigates-and-humans-decide-5a2bc33b464a

Built With

  • alibaba-cloud-model-studio
  • css3
  • dashscope
  • fastapi
  • fastmcp
  • html5
  • httpx
  • javascript
  • mcp
  • npi-registry-api
  • psychology-today
  • pydantic
  • pytest
  • python
  • qwen-max
  • qwen-plus
  • server-sent-events
  • twilio
  • twiml
  • uvicorn
Share this project:

Updates