Sonar-Drishti

AI-powered sonar vision that helps operators find hidden seabed hazards faster with OpenCV 5 and AWS.

Comment

DRISHTI — Sonar Edge Inference

Inspiration

Lost fishing gear, wrecks, pipelines, and other seabed hazards can be difficult to spot in side-scan sonar. Survey teams may need to review large collections of sonar tiles, where speckle, shadows, and seabed texture can resemble objects of interest.

DRISHTI explores how computer vision can help analysts prioritize that review. We built a detector for sonar imagery and packaged its ONNX version in a CPU-based edge service. The goal is practical: make it easy to submit a tile, see the model's candidate detections, and inspect the image regions that triggered them.

What it does

The deployed demo accepts one PNG or JPEG sonar image (up to 10 MB) and returns candidate detections with class labels, confidence scores, bounding boxes, and runtime information. The browser displays the boxes over the uploaded image. The service also exposes a health endpoint and interactive API documentation.

The live edge demo runs on AWS. Uploaded image bytes are processed by the inference service; when result storage is enabled, only the JSON result is written to S3. The deployed single-tile flow does not attach geographic coordinates.

The React analyst dashboard

The repository also includes a separate React 18 + Vite dashboard for the fuller local survey workflow. It is designed for multi-file uploads, live per-tile job updates over WebSockets, a map of geotagged detections, an analyst review queue with confirm/reject actions, and JSON, CSV, or GeoJSON exports. When navigation data is supplied to the local processing workflow, detections can be projected to search-area coordinates for map review.

This React dashboard is part of the project, but it is not the page shown by the public AWS URL. It depends on the Django API, PostgreSQL, Redis, and Celery worker; those services are described in the repository's local setup guide. We deployed the smaller FastAPI + ONNX edge app because it is a lighter, self-contained demo. The full-stack local setup has not been smoke-tested as part of the current deployment verification, so treat it as a separate workflow that requires setup rather than a live hosted dashboard.

How we built it

  • Model: A YOLO-family detector trained for sonar object detection and exported as an FP32 ONNX model. The edge container loads the ONNX artifact rather than a PyTorch checkpoint.
  • Image processing: OpenCV handles image decoding, grayscale conversion, Lee filtering, and CLAHE contrast enhancement before inference.
  • Inference: ONNX Runtime runs on its CPU execution provider. The service filters detections and returns boxes and scores through a FastAPI API.
  • Web interface: A lightweight static HTML/JavaScript page served by the same FastAPI container provides upload, inference, and visual review.
  • AWS deployment: Amazon ECR stores the image; Amazon ECS Express Mode on Fargate runs the service; Amazon S3 stores optional result JSON; and Amazon CloudWatch stores service logs.
  • React dashboard: A separate React 18 + Vite + React-Leaflet frontend provides the local operator experience: multi-file jobs, WebSocket progress, geotagged map review, confirm/reject controls, and report exports. It talks to the Django API and is not included in the lightweight AWS edge container.
  • Two deployment modes: Run the edge UI/API locally with the Docker instructions in the repository README, or set up the full Django/PostgreSQL/Redis/Celery stack plus the Vite dashboard using the local setup guide. Only the edge UI/API is deployed to AWS.

Results

We evaluated the shipped ONNX model on the pinned public test split of 700 sonar tiles. The following are fixed-threshold precision and recall at confidence 0.25 and IoU matching threshold 0.50. They are not mAP, and they have not been calibrated into probabilities.

Class Precision Recall True positives False positives False negatives
Submarine pipeline 96.6% 96.6% 168 6 6
Shipwreck 64.3% 34.3% 180 100 345
Ghost net 100.0% 100.0% 120 0 0
Mine cylinder 59.4% 46.3% 38 26 44

The ghost-net test examples are synthetic-on-synthetic evidence and do not establish performance on real-world ghost nets. Shipwreck site-disjointness has not been fully audited. Evaluation took an observed 134.7 ms per tile in the test container, including image decode, inference, and matching; the host hardware was not recorded, so this is not a hardware-independent latency claim.

Earlier project notes included results from an 850-tile evaluation and calibration figures. We could not reproduce those figures from the pinned dataset, whose test split contains 700 tiles, so we do not present them as current results.

Challenges we ran into

  • Sonar texture, acoustic shadows, and noise can look like objects. Preprocessing and model output thresholds both affect which candidates are shown.
  • Public labelled data is limited and may not represent open-water surveys. We pinned the dataset revision and audited the test split so the reported evaluation can be tied to a specific set of images and labels.
  • Shipping an ONNX-based runtime meant keeping the inference container separate from the larger training and dashboard stack.
  • Turning a model into a usable demo required more than an endpoint: the edge service needed an upload page, visualized boxes, health reporting, and a deployable AWS container.

Accomplishments that we're proud of

  • We turned the trained sonar detector into a lightweight, torch-free ONNX inference service and deployed it on AWS ECS Express Mode.
  • We built a browser upload interface that displays detections over the sonar image and shows the active model and runtime details.
  • We kept the full React analyst dashboard in the project as a separate local workflow, with live job progress, map-based review, analyst decisions, and report exports.
  • We audited the pinned public test split and published reproducible per-class precision and recall instead of relying on older, unverified headline metrics.

What we learned

  • Fixed-threshold results by class reveal meaningful differences that a single headline number would hide.
  • CPU inference and a compact ONNX runtime make a straightforward edge deployment possible, but latency still needs to be reported with the hardware and measurement method.
  • A detection score is a model score, not automatically a calibrated probability. We need more evaluation before making confidence-calibration claims.
  • A useful sonar tool should support analyst review. DRISHTI presents detections as candidates, not confirmed findings.

Limitations and responsible use

  • Results are candidate detections for human review, not confirmed objects or enforcement evidence.
  • Recall varies substantially by class; the current shipwreck and mine-cylinder results show that the model can miss many labelled examples.
  • The ghost-net result is based on synthetic-on-synthetic examples and must not be interpreted as field performance.
  • The evaluation does not yet establish generalization across independent survey sites or sonar systems.
  • The deployed one-image edge demo does not perform geolocation. Position estimates in the broader project require survey navigation data and should be treated as search areas.
  • The public demo accepts unauthenticated image uploads, is limited to 10 MB per image, and runs with a maximum of one ECS task.

What's next for Sonar-Drishti

  • Recompute mAP and calibration on the pinned public test split and publish the complete evaluation protocol.
  • Benchmark preprocessing and inference separately on a recorded CPU or edge device.
  • Validate on site-disjoint, real survey data and document dataset licenses and attribution.
  • Add authentication and usage controls before using the public service beyond a demonstration.
  • Capture a short demo showing both detections and a no-detection sonar tile.

Built with

Python, FastAPI, YOLO, ONNX, ONNX Runtime, OpenCV 5, NumPy, HTML, CSS, JavaScript, Docker, Amazon ECR, Amazon ECS Express Mode on Fargate, Amazon S3, and Amazon CloudWatch.

Links

Team

Team StrawHats. Dataset and model attribution details are maintained in the repository documentation; confirm the final credit and license wording before submission.

Built With

Share this project:

Updates

Submission history