Turning ILI Runs into Actionable Integrity Insights

Inspiration

Pipeline integrity decisions rely on comparing In-Line Inspection (ILI) runs collected years apart.
In practice, these datasets cannot be directly compared due to odometer drift, vendor differences, renamed or missing features, and sensor uncertainty.

Today, integrity engineers manually align these runs — a process that can take weeks, varies analyst to analyst, and can introduce false corrosion growth or missed defects.

We were inspired by a simple but critical question:

How can we turn misaligned ILI data into physically honest, defensible integrity insights—automatically?

This project was motivated by real-world integrity workflows where alignment errors matter as much as corrosion itself, directly impacting safety, compliance, and excavation decisions.

What We Built

We built a physics-aware, engineering-driven system that:

Aligns multiple ILI runs to a common physical reference frame
Matches corrosion anomalies across years with explicit confidence scoring
Calculates annualized corrosion growth rates
Identifies interaction zones where clustered defects increase failure risk
Converts thousands of anomalies into a small, actionable set of dig locations

The system transforms raw inspection data into clear integrity decisions, not just matched rows.

How We Built It

Our approach was structured into three engineering phases.

Phase 1 — Reference Alignment (“Common Ruler”)

ILI odometer error accumulates gradually, meaning a simple shift correction is incorrect.
We applied a rubber-sheeting alignment, where distance correction scales along the pipeline.

Used rare, fixed features (valves, tees, taps) as anchors
Matched anchors using elevation fingerprints, even when names differed
Interpolated correction between anchors to remove drift

This produced a physically consistent alignment where corrected pipeline lengths converged.

Phase 2 — Anomaly Matching & Growth Engine

We focused only on metal loss features, since growth prediction is meaningful only for corrosion.

Each anomaly was matched using multiple physical constraints:

Corrected distance (± tolerance)
Clock position (circular math)
ID/OD consistency
Dimensional sanity checks

We introduced confidence scoring, starting from 100% and applying penalties for mismatches, allowing the algorithm to say “I’m not sure” instead of forcing bad matches.

Corrosion growth was computed as annualized depth growth

This normalization prevents misleading conclusions from total growth alone.

Phase 3 — Interaction Zone Analysis

Single pits rarely fail pipelines — clusters do.

We applied DBSCAN clustering to group nearby corrosion features into interaction zones:

No predefined cluster count
Noise-resistant
Aligned with corrosion physics

This reduced thousands of anomalies into a few hundred high-risk dig locations.

Challenges We Faced

Odometer drift increased non-linearly over long distances
Feature names and identifiers changed between vendors and runs
Joint numbers failed due to pipe replacements
Sensor tolerance produced apparent “negative growth”
Anomalies split, merged, or disappeared between inspections

Instead of hiding these issues, we designed the system to expose uncertainty explicitly.

What We Learned

Alignment is the hardest problem in ILI analytics — not growth modeling
Confidence scoring is more valuable than forced precision
Average corrosion rates look safe; outliers drive failures
Growth distributions reveal physics when alignment is correct
Explainability is essential in safety-critical engineering systems

Most importantly, we learned that integrity analytics must respect physical reality, not just data similarity.

Looking Ahead

This project is a foundation for:

Circumferential (2D) clustering
Automated dig-sheet generation
Bayesian corrosion growth modeling
Pressure re-rating simulations
Digital pipeline integrity twins

Our goal is to help pipeline operators move from manual alignment and reactive decisions to predictive, data-driven integrity management — safely and transparently.