LiDAR Forensics

LiDAR Forensics distinguishes sensor-specific LiDAR stalls from recording-wide gaps, exposes timing evidence, and generates conservative findings without proprietary data.

Comment

Inspiration

During a real mobile LiDAR survey, I discovered a major hole in the processed point cloud. The standard processing pipeline stopped at the failure and treated the remaining data as permanently lost.

I did not accept that result as the final technical answer.

I investigated the RAW recording stream by stream to determine what had actually stopped, what had continued, and whether usable measurements still existed after the processing break.

The analysis revealed a 3.404-second LiDAR-only publication stall. During that interval, IMU, motor, encoder, GPS/GNSS, and recorder activity continued. The recording had not stopped as a whole.

Some LiDAR measurements were genuinely absent during the stall, and I did not synthesize or invent them. However, real LiDAR measurements still existed in the RAW recording after the trajectory produced by the stock software had stopped.

Using the surviving sensor measurements and trajectory evidence, I reconstructed usable point-cloud geometry beyond the original processing break. The conservative reconstructed result contained more than 100 million usable points.

The same LiDAR-only failure signature was later identified in multiple recordings from two physical scanners.

That investigation became LiDAR Forensics: a reusable diagnostic application for understanding multistream sensor failures, separating confirmed evidence from engineering hypotheses, and determining whether a damaged recording may still contain recoverable data.

What it does

LiDAR Forensics analyzes synchronized event metadata from LiDAR, IMU, motor, encoder, GPS/GNSS, and recorder streams.

It detects and distinguishes:

  • normal multistream recording;
  • LiDAR-only publication stalls while companion telemetry remains active;
  • recording-wide gaps affecting all observed streams;
  • stale sensor timestamps after stream resumption;
  • post-stall catch-up bursts;
  • repeated interruptions;
  • premature stream termination.

The application presents:

  • recording continuity;
  • LiDAR relative availability;
  • observed sensor streams;
  • an interactive multistream timeline;
  • incident classification;
  • a structured Finding register;
  • machine-readable evidence;
  • engineering interpretation;
  • recommended next tests;
  • exportable investigation results.

Each finding separates:

  • confirmed observations;
  • engineering assessment;
  • supporting evidence;
  • unresolved questions;
  • recommended validation steps.

Results can be exported as CSV, JSON, standalone HTML, and Markdown.

The public application focuses on diagnosis, evidence packaging, and failure classification. Full point-cloud reconstruction is not performed inside the browser application because recovery depends on which real measurements, timestamps, calibration data, and trajectory information survived in a specific recording.

How it was built

The backend uses Python, FastAPI, and Pydantic.

The detection pipeline applies deterministic rules to normalized multistream event data. It evaluates stream continuity, message timing, inter-message gaps, timestamp behavior, stream resumption, and relationships between primary and companion sensors.

The application includes normalized CSV and JSON adapters, with optional ROS1 BAG and MCAP support through open-source libraries.

The frontend uses static HTML, CSS, JavaScript, and native SVG. The multistream timeline is rendered directly in the browser without requiring a separate visualization framework.

Three deterministic demonstration datasets are included:

  • a normal multistream recording;
  • a 3.4-second LiDAR-only publication stall;
  • a 3.4-second recording-wide gap.

These scenarios make it possible to reproduce the diagnostic behavior without distributing the original field recording.

The repository also contains automated tests for:

  • detection rules;
  • input adapters;
  • API behavior;
  • exports;
  • interface contracts;
  • desktop and mobile presentation.

How Codex and GPT-5.6 Sol were used

The core application was developed in Codex using GPT-5.6 Sol, primarily with Very High reasoning effort and Ultra reasoning for selected complex tasks.

Codex helped transform the original engineering investigation into a structured application by implementing:

  • the FastAPI backend;
  • normalized data adapters;
  • deterministic detection rules;
  • demonstration datasets;
  • the browser interface;
  • the SVG timeline;
  • evidence exports;
  • automated tests;
  • documentation;
  • release packaging.

The human engineer supplied:

  • the original field failure;
  • the RAW-stream investigation method;
  • the sensor-role model;
  • the failure taxonomy;
  • the evidence requirements;
  • the engineering interpretation;
  • the validation criteria;
  • the reconstruction constraints;
  • the boundaries between confirmed facts, hypotheses, and unsupported claims.

GPT-5.6 did not independently discover the physical failure and was not the source of the field measurements. It was used as an engineering development partner to help convert the investigation into a tested and reproducible software product.

Challenges

The first challenge was working with an undocumented multistream RAW recording.

A visible hole in a point cloud does not immediately reveal whether the cause is:

  • missing physical measurements;
  • a stopped sensor stream;
  • a recorder failure;
  • a timestamp discontinuity;
  • a driver or publisher stall;
  • frame-assembly behavior;
  • a transport problem;
  • a trajectory failure;
  • or a processing limitation.

The second challenge was preserving the distinction between observation and cause.

A 3.404-second LiDAR message gap proves that no LiDAR messages were recorded during that interval. Continued IMU, motor, encoder, GPS/GNSS, and recorder activity proves that the complete recording did not stop.

However, timing evidence alone does not prove whether the initiating mechanism was inside firmware, a driver, frame assembly, publishing, networking, power, or hardware.

The third challenge was defining availability metrics correctly.

Recording continuity and LiDAR relative availability use different denominators. A recording-wide gap and a LiDAR-only stall must not be represented as the same type of failure or collapsed into one misleading percentage.

The final challenge was turning a one-off forensic investigation into a reproducible application without pretending that every damaged recording can be recovered.

Accomplishments

  • Identified a 3.404-second LiDAR-only publication stall in a real mobile LiDAR recording.
  • Confirmed that IMU, motor, encoder, GPS/GNSS, and recorder activity continued during the LiDAR interruption.
  • Confirmed that real LiDAR measurements survived after the trajectory produced by the stock software had stopped.
  • Reconstructed usable point-cloud geometry beyond the original processing break without inventing missing measurements.
  • Produced a conservative reconstructed result containing more than 100 million usable points.
  • Identified the same LiDAR-only failure signature in multiple recordings from two physical scanners.
  • Converted the forensic investigation method into a working public application.
  • Made LiDAR-only and recording-wide failures visibly and numerically distinct.
  • Separated primary interruption incidents from related timestamp and catch-up findings.
  • Added CSV, JSON, standalone HTML, and Markdown exports.
  • Created deterministic normal, LiDAR-stall, and recording-wide-gap demonstrations.
  • Reached 24 passing automated tests.
  • Published a reproducible GitHub release.

What was learned

Data that standard software cannot process is not necessarily physically absent.

A processing pipeline may stop using a recording even though valid measurements still exist after the failure point.

Multistream continuity is often more informative than a single-stream error message. When IMU, motor, encoder, GPS/GNSS, and recorder activity continue while LiDAR publication stops, the failure signature becomes much narrower.

It is also important to separate three different questions:

  1. What was directly observed?
  2. What mechanism is most consistent with the evidence?
  3. What additional test would be required to prove the cause?

The project also showed that recording continuity and LiDAR relative availability describe different engineering properties and should not be reduced to one percentage.

What is next

Future development could include:

  • configurable stream-role profiles for different sensor platforms;
  • direct packet-sequence diagnostics;
  • clock-domain alignment analysis;
  • queue and frame-assembly telemetry;
  • firmware and driver A/B comparison tools;
  • packet-capture correlation;
  • streaming analysis for very large recordings;
  • recovery-readiness scoring;
  • support for additional robotics and mobile-mapping formats;
  • optional point-cloud reconstruction for authorized recordings with sufficient surviving data.

A future reconstruction module would only operate when the required measurements, timestamps, calibration information, and trajectory evidence are available. The application would not claim that every damaged recording can be recovered.

Public demonstration data

The public repository uses deterministic demonstration datasets instead of distributing the original field recording.

The public application reproduces the diagnostic method, failure classification, evidence structure, timeline behavior, and reporting workflow. The precise initiating mechanism behind the original physical failure has not yet been proven.

Built With

Share this project:

Updates