IndustrialPilot — Autonomous Factory Agent

An autonomous AI agent that diagnoses factory equipment faults, takes real corrective action via simulated SCADA/PLC controls, and escalates to technicians only when it can't safely act alone.

Comment

IndustrialPilot — Autonomous Factory Incident Response Agent

Inspiration

I came into this hackathon already having worked on industrial AI before — a previous project of mine used Gemini and Dynatrace to predict motor failures before they happened. That experience left me with one frustration: most "AI monitoring" tools stop at the alert. They tell you something is wrong, maybe even diagnose it, but then they just sit there. A human still has to read the alert, decide what to do, and go do it — every single time, even for the same routine fault that's happened fifty times before.

I wanted to build something that goes one step further: an agent that doesn't just detect a problem, but actually acts on it the way a competent control-room operator would — and knows, precisely, when it should not act and hand off to a human instead. That boundary felt like the real engineering challenge. Anyone can wire an LLM to a sensor feed. Building an agent that knows the difference between "I can fix this safely" and "this needs a person" is what makes it production-grade instead of a toy demo.

That's how IndustrialPilot started: Track 4, Autopilot Agent, built around one core question — what should an AI be allowed to do alone in a factory, and what should it never be allowed to do alone?

What It Does

IndustrialPilot is an autonomous agent that watches over 8 simulated industrial machines — three motors, two pumps, a compressor, a conveyor, and a boiler — each with live sensor readings for temperature, vibration, current, pressure, flow rate, or speed depending on the equipment.

When a reading crosses its threshold, the agent:

  1. Reads the live sensor history before forming any opinion
  2. Diagnoses the root cause and scores its own confidence from 0 to 1
  3. Takes exactly one of three actions, chosen by strict, code-enforced rules:
    • If it's electronically fixable and confidence is high enough, it sends a real remediation command — reducing motor load through a simulated VFD, opening a pressure relief valve, cutting a boiler's firing rate through a simulated BMS
    • If the fault is mechanical — a bearing, a vibration pattern, a slipping belt — it always creates a maintenance work order, no matter how confident it is, because no software command can repair a physical bearing
    • If confidence is low, the fault is electrical, or the situation is hazardous, it escalates to a human technician by email and Slack, and can trigger a full emergency shutdown that cuts power to the unit

Every action is reflected on the live sensor reading itself — fixing a problem actually drops the temperature, the current, the pressure, the same way it would on a real control system. Every decision, every tool call, and every outcome is written to a permanent audit trail that a human can review, filter, and export to CSV.

The whole thing runs as a live dashboard with five tabs: a control center for firing alerts and watching the agent reason in real time, a sensor status overview, a full incident history, an equipment reference sheet, and a settings page for configuring who gets notified.

How I Built It

I built this collaboratively with Claude, working file by file rather than generating the whole thing at once. The backend is Python and FastAPI, with the agent's reasoning loop calling Qwen Cloud's qwen-flash model through tool-calling — the agent doesn't just produce text, it actually invokes functions like reduce_motor_load, create_maintenance_work_order, or escalate_to_operator, and each of those functions changes real, shared server state.

That shared state was one of the trickiest design decisions. Early on, the browser and the server each kept their own copy of sensor readings, and they drifted apart — the dashboard would show one number while the agent reasoned about a different one. I rebuilt the sensor layer so the server is the single source of truth: the browser only ever displays what the server reports, and every fix the agent makes is written back to that same shared state before the dashboard re-renders. That single change fixed a cluster of bugs where the agent appeared to "fix" something and the screen wouldn't move.

The permission boundary — what the agent can and cannot do alone — isn't just a prompt instruction. It's enforced directly in code. If the model tries to mark an incident "resolved" without having called a real remediation tool, the system forces an escalation instead. If a remediation tool was called but the model's own confidence came back under 80%, the system still escalates — an unverified fix is never silently treated as done. I added this after noticing the agent could otherwise talk itself into a false "auto-resolved" status, which is exactly the kind of failure that would be unacceptable on a real factory floor.

I also rebuilt every remediation action to follow real physical cause-and-effect instead of an arbitrary percentage drop. Reducing a motor's load lowers its current draw directly — the actual electrical consequence of a VFD speed command — and temperature follows afterward at a slower rate, the way thermal lag actually behaves. Cutting a boiler's firing rate drops its temperature first and its pressure second, because that's the order those two values genuinely respond in a real burner management system.

Once the logic was solid, I deployed it to an Alibaba Cloud ECS instance running Ubuntu — cloning the repository, configuring the environment, and running it persistently so it's reachable at all times during judging.

Challenges I Faced

Server vs. browser state drift. As above — this was the single hardest bug to track down, because it didn't look like a bug. It looked like the agent "doing nothing." The fix required treating the backend as authoritative and rebuilding several frontend functions to stop assuming local state was correct.

Stopping the agent from lying to itself. Language models are good at producing a confident-sounding summary even when they haven't actually done anything. Getting "AUTO-RESOLVED" to mean something real — backed by an actual tool call and an actual confidence check — took moving that logic out of the prompt and into hard, unconditional code paths.

Making the physics honest. It would have been easy to have every "fix" just nudge a number down by 20%. Instead, each remediation tool had to map to the real mechanism it represents — a pressure-relief valve changes pressure, not temperature; a firing-rate cut changes temperature first; a bearing problem can't be touched by software at all. Getting these chains right meant thinking through what each piece of industrial equipment actually does, not just what would look convincing on a dashboard.

Working around a real token budget. Qwen Cloud's free tier ran out partway through development, and the $40 hackathon credit voucher never arrived despite registering correctly. I switched to qwen-flash for its lower cost, and restructured testing so that logic changes were verified offline with mock function calls first — only spending real API tokens once I was confident a change was correct, instead of debugging live against the model.

Deploying for the first time. This was my first time provisioning and deploying to a cloud server. Getting the security group rules right so the app was actually reachable, learning the difference between localhost and a public IP, and learning to run the process detached so it survives closing the browser were all new — and all things that would have silently broken the deployment if missed.

What I Learned

The biggest lesson was that autonomy without boundaries isn't useful — it's a liability. The interesting engineering problem in this project was never "can the AI call a function." It was "how do I make sure the AI only calls the right function, in the right circumstances, and never quietly pretends it succeeded when it didn't." That meant writing rules that don't trust the model's own self-report, and verifying outcomes in code rather than taking the model's word for it.

I also learned, the hard way, that local testing and cloud deployment are different problems. Code that runs perfectly on a laptop can fail silently on a server for reasons that have nothing to do with the code itself — network ports, process detachment, public versus private IP addressing. None of that shows up until you actually deploy.

What's Next

The architecture generalizes well beyond these eight simulated machines. The same pattern — diagnose, score confidence, choose between autonomous action, mandatory human handoff, or emergency shutdown — applies to any industrial system with sensor data and a defined set of safe remote actions. Natural next steps include connecting it to a real OPC-UA or Modbus interface instead of a simulation, expanding the equipment library to cover more machine types, and building a proper multi-technician assignment system so escalations route to whoever is actually on shift.

Built With

  • alibaba
  • cloud
  • elastic-compute-service
  • fastapi
  • html/css/javascript
  • python
  • qwen-cloud-(qwen-flash)
  • sqlite
  • websocket
Share this project:

Updates