Facility Incident Investigator

An autonomous AI system that investigates hard-to-diagnose building incidents, finds the real root cause, triggers repairs, and verifies the issue is actually resolved.

Comment

Inspiration

I built Facility Incident Investigator after dealing with a real issue in my own apartment. A motor was making a lot of noise, and it took almost a week to resolve because the problem had to move through different people and systems.

That made me think about a broader problem in building operations: the symptom someone reports is often not the actual cause.

Modern buildings already have sensors, maintenance records, alarms, equipment data, and topology. The hard part is connecting all of that information when something goes wrong.

What it does

Facility Incident Investigator has two parts:

  1. A realistic eight-floor building simulator with sensors, equipment, maintenance history, topology, and hidden physical failures.
  2. A Gemini-powered multi-agent system that investigates incidents using only the operational evidence available to a real facilities team.

The agents cannot see the simulator's hidden answer.

The Incident Commander dynamically chooses what to investigate next and can call specialist agents for Safety, BMS/sensor analysis, CMMS/maintenance history, and spatial/building topology.

New evidence can strengthen, weaken, or reject hypotheses and change the investigation path.

Once enough evidence is collected, the system creates a precise repair work order, waits for external technician work, resumes later, and verifies post-repair telemetry before closing the incident.

Why this is challenging

A reported symptom can be many causal steps away from the real failure.

Our flagship demo uses a 10-hop cross-floor causal chain. Occupants on Floor 7 report flickering lights and a hot-plastic odor, while the real failure begins at an upstream electrical connection on Floor 1.

Solving it requires correlating sensor history, maintenance records, topology, equipment relationships, and timing across multiple systems.

The agents must independently reconstruct the same cause that the simulator secretly injected.

How we built it

The project uses:

  • Gemini 3.5 Flash
  • Google Agent Development Kit (ADK)
  • Vertex AI
  • Google Cloud Run
  • Firebase Firestore
  • Pub/Sub
  • Cloud Tasks
  • FastAPI
  • React
  • TypeScript
  • Python

The workflow is designed for long-running tasks. Each bounded step is checkpointed, allowing the system to wait, retry after failures, survive process restarts, and resume without duplicating evidence, work orders, or repair actions.

The cloud deployment uses separate Cloud Run services for the API, workflow worker, and simulator. Operational state is stored separately from private simulator ground truth.

Challenges

The biggest challenge was making the system reliable across long-running physical workflows.

A repair may happen minutes or hours after the original investigation, so correctness cannot depend on one long model call or one in-memory server process.

We built durable checkpoints, deterministic event IDs, retries with backoff, idempotent repair actions, startup recovery, and independent post-repair verification.

Another challenge was making the simulator credible. Hidden failures must change real catalog assets and sensor points, while agents must never be able to access the injected answer.

What we learned

Long-running agents are primarily a state and event-delivery problem, not just a model-context problem.

Gemini is most useful for deciding what evidence to gather and what to investigate next. Persistence, safety boundaries, retries, duplicate prevention, and closure rules are better enforced deterministically.

We also learned that complex physical-world incidents are a strong agentic use case because no single system contains the full answer.

What's next

The next step is integrating the same architecture with real BMS and CMMS systems.

The same durable investigation pattern could also apply to industrial equipment failures, infrastructure maintenance, logistics incidents, and other operational workflows where the cause is uncertain and resolution spans multiple systems and people.

Built With

Share this project:

Updates

Submission history