Industrial Inspection & Worker Safety Monitoring System

An autonomous system that inspects pipelines using vision and gas sensors while monitoring workers for distress or unconsciousness, providing real-time alerts and mapping hazards to prevent accidents.

Comment

Inspiration

Every year, thousands of workers enter confined spaces like underground pipelines, industrial ducts, chemical transport lines, and sewer conduits with little more than a handheld gas meter. According to OSHA, confined space incidents cause over 90 fatalities annually in the US alone, with toxic gas exposure and oxygen displacement as the leading causes. Beyond the human cost, traditional pipeline inspection using manual walkdowns or mechanical pigging costs the oil and gas industry over $6 billion per year globally, yet slow and diffuse leaks regularly go undetected for months. We kept asking why humans were still being sent into environments we cannot reliably measure. This project is our answer: an autonomous, sensor-fused, AI-powered drone that enters where humans should not, and comes back with a complete picture of what is happening inside.

What it does

The system is an autonomous drone-based pipeline inspection platform that simultaneously performs multi-gas leak detection, 3D spatial leak mapping, GPS-denied navigation, and real-time worker safety monitoring. The drone carries a nine-sensor payload covering LPG, methane, hydrogen, carbon monoxide, ammonia compounds, alcohol vapors, PM2.5 particulate matter, temperature, humidity, barometric altitude, and full inertial motion. All sensor data is streamed live to a ground station dashboard that displays a live annotated camera feed, a 3D leak intensity map with distance and altitude axes, a 2D leak trend chart, a full numeric sensor panel, a drone-over-pipe simulation that responds to live IMU data, and a top-down heat map of the drone's traveled path. A leak trend forecasting algorithm computes the rate of change of gas concentration and predicts leak intensity one meter ahead of the drone, providing early warning before the drone physically reaches the hazard zone. The entire dashboard is also served over a local web server, allowing any supervisor on the network to monitor the mission live from a browser with no additional software. If a worker is detected lying motionless in an elevated-gas environment, the system immediately triggers a DANGER alert with location tagging.

How we built it

The architecture follows a three-tier distributed design. At the first tier, an ESP32 microcontroller aggregates readings from all nine sensors at 10 Hz and serializes them into structured ASCII packets transmitted over UART at 115200 baud. This separation keeps sensor acquisition latency under 2 ms and completely decouples timing from the compute layer. At the second tier, a Raspberry Pi 3B handles all processing: it parses incoming packets, applies per-sensor exponential moving average filtering with a smoothing factor of 0.15, runs a sliding window linear regression over 30 samples to compute the leak trend slope and one-meter-ahead prediction, and builds the 3D spatial map by fusing optical flow distance estimates with BMP280 barometric altitude. For navigation in GPS-denied environments, the Pi fuses optical flow, IMU dead reckoning, and pipeline centerline alignment through a complementary filter, with a Mahony quaternion filter suppressing gyroscope drift. The vision system runs two YOLOv8n models: one for pipeline detection and tracking, and one for worker presence and posture classification. Both models were INT8 post-training quantized to reduce inference time from roughly 850 ms to 220 ms on the Pi, achieving approximately 4.5 FPS at 320x240 resolution. At the third tier, the ground station renders all six dashboard views simultaneously and serves them over a TCP socket web server for remote access.

Challenges we ran into

MQ-series gas sensors exhibit significant cross-sensitivity and thermal drift, making raw readings unreliable without careful calibration. We performed a two-point calibration per sensor using the power-law model C=a⋅(RS/R0)bC = a \cdot (R_S / R_0)^b C=a⋅(RS​/R0​)b to establish accurate concentration estimates, and used the EMA filter alongside a rate-of-change alert trigger at three standard deviations to catch genuine leak events regardless of baseline drift. Running YOLOv8 inference on the Raspberry Pi at full precision gave approximately 1.2 FPS, which was completely unusable. Reaching real-time performance required stacking model selection (YOLOv8 nano at 3.2 million parameters), INT8 quantization, frame skipping with interpolation, and resolution reduction to 320x240. IMU dead reckoning accumulates positional error quadratically: raw double integration drifted 1.2 meters in just 10 seconds during bench testing. We resolved this by promoting optical flow to the primary navigation source, applying the Mahony filter for attitude estimation, and using pipeline wall alignment as a periodic position anchor. Finally, synchronizing sensor data at 10 Hz, vision at 4.5 FPS, and dashboard rendering at 30 FPS required a timestamp-aligned ring buffer with a 500 ms lookahead on the Pi to prevent gas readings from being spatially misaligned with the video frame.

Accomplishments that we're proud of

We are proud of building a fully integrated system where every layer, sensors, microcontroller, edge compute, computer vision, and live dashboard, works together in real time on affordable consumer hardware costing under $100 in total. In our live demonstration the system correctly identified a multi-gas hazard scenario with LPG at 2896 ppm, CH4 at 2415 ppm, and CO at 2895 ppm, localized the leak zone spatially to a distance window of 85 to 95 meters on the 3D map, and accurately predicted the next-meter concentration of 2987.7 ppm from a trend slope of 91.7 ppm/m. Running YOLOv8 inference at usable frame rates on a Raspberry Pi through quantization and optimization, without any dedicated AI accelerator, was a particular technical achievement. We are also proud of the dashboard itself: the drone-over-pipe simulation responding live to IMU data and the heat-map mini-map together give operators a spatial situational awareness that no handheld instrument can provide.

What we learned

Edge AI is achievable on constrained hardware but demands that model selection, quantization, resolution, and frame rate be treated as first-class engineering decisions rather than afterthoughts. Sensor fusion consistently outperforms any single modality: optical flow alone drifts, IMU alone drifts faster, and barometric altitude alone is noisy, but their combination weighted by confidence is robust enough to navigate a real pipeline. The dashboard is as important as the underlying robotics. Operators interact with the interface, not the firmware, and the quality of that interface directly determines whether the system is useful in practice. Perhaps most importantly, hardware engineering is qualitatively different from software engineering. A UART framing error that appears only after 45 minutes of continuous uptime, or an MQ-7 sensor that re-drifts every time ambient temperature shifts by 3 degrees, cannot be fixed with a recompile. Those problems require patience, measurement, and iterative physical debugging.

What's next for Industrial Inspection & Worker Safety Monitoring System

The immediate next step is integrating ORB-SLAM3 on an NVIDIA Jetson Nano to replace dead reckoning with a proper globally consistent 3D map, enabling true autonomous path planning along the pipeline without any external reference. We plan to add an MLX90640 thermal camera for hot-spot detection and human detection that is independent of ambient light and gas conditions. On the AI side, we want to train a multi-label classifier on the nine-sensor array signatures to chemically distinguish gas species from each other, rather than relying on separate sensors for each gas. For communications, integrating LoRaWAN will extend operational range to several kilometers and enable multi-drone mesh networking across large pipeline networks. Longer term, the visual feed will be used to train a crack and structural defect detection model, making the system sensitive to both chemical and physical pipeline degradation simultaneously.

Built With

  • 01:47claude-responded:-python
  • bmp280
  • c++-(esp32-firmware)
  • dht11
  • esp-idf
  • esp32
  • fritzing-(circuit-design)
  • gp2y1010au0f
  • int8-post-training-quantization
  • luca?python
  • lucas-kanade-optical-flow
  • mahony-complementary-filter
  • matplotlib
  • mpu6050
  • mq-series-gas-sensors
  • numpy
  • opencv
  • probabilistic-hough-transform
  • pytorch
  • raspberry
  • raspberry-pi-os
  • tcp-socket-server
  • uart-serial-protocol
  • yolov8-(ultralytics)
Share this project:

Updates