FloodLeans AI

FloodLens AI analyzes uploaded road images and recorded videos to detect potential urban flooding using OpenCV 5, YOLO11 segmentation, and temporal verification.

Comment

Inspiration

Urban flooding is a serious challenge for cities, affecting mobility, damaging infrastructure, and putting lives at risk. Many urban areas already have road cameras, but manually reviewing large amounts of footage is time-consuming.

FloodLens AI was created to explore how computer vision and artificial intelligence can help identify potential urban flooding from visual information, making analysis more accessible, interpretable, and useful.

Our goal is to develop a foundation for future intelligent urban monitoring systems.

What it does

FloodLens AI is a computer vision application that analyzes uploaded road images and prerecorded videos to identify potential flooding.

The system provides:

  • Flood segmentation: A YOLO11n-seg model identifies potential flooded regions in road scenes.
  • Visual explanations: OpenCV 5 generates segmentation masks and cyan overlays highlighting detected regions.
  • Temporal verification: Video analysis samples approximately one frame per second and uses five-frame sliding windows, requiring at least three positive samples for temporal confirmation.
  • Area estimation: Calculates the percentage of image pixels classified as flooded.
  • Experimental risk analysis: Produces heuristic risk categories based on visual area, confidence, and temporal evolution when flooding is temporally confirmed.
  • Interactive dashboard: Displays analysis results, visual previews, temporal metrics, and detection status.
  • AWS integration: Supports containerized execution on AWS and optional storage of uploaded media and analysis results in Amazon S3.

The application currently processes uploaded images and recorded videos, rather than continuous live camera streams.

How we built it

FloodLens AI combines computer vision, deep learning, backend development, and cloud computing.

Computer Vision and AI: We used OpenCV 5 for image decoding, video frame extraction, segmentation mask processing, and visualization. A fine-tuned YOLO11n-seg model performs flood-region segmentation using PyTorch and Ultralytics.

Temporal Analysis: The system evaluates consecutive video samples to distinguish isolated detections from persistent predictions. A sliding window of five samples requires at least three positive detections to confirm a video-level flood candidate.

Backend: We developed a REST API with Python and FastAPI, providing endpoints for image analysis, video analysis, health checks, and optional analysis history.

Frontend: We implemented an interactive web dashboard that displays original media, segmentation previews, detection results, and temporal visualizations.

Cloud Infrastructure: We containerized the application with Docker and successfully validated an end-to-end deployment using Amazon ECR, Amazon ECS/Fargate, and Amazon S3.

The cloud deployment was tested successfully and subsequently scaled down to reduce operating costs.

Challenges we ran into

One of our biggest challenges was distinguishing actual urban flooding from visually similar conditions.

Wet pavement, rain, reflections, ordinary puddles, shadows, and nighttime lighting can cause incorrect predictions.

We experimented with multiple training datasets and hard-negative examples to improve model robustness. However, an additional hard-negative training experiment did not improve the aggregate development-set classification results, so we retained our original MultiDomain checkpoint.

Another challenge was making video predictions more consistent. We implemented temporal verification to reduce isolated false detections, although persistent visual errors can still produce false positives.

Finally, integrating the computer vision pipeline with a web interface, Docker, and AWS required resolving challenges involving dependencies, inference execution, cloud configuration, and reproducibility.

Accomplishments that we're proud of

We developed a functional end-to-end research prototype that combines segmentation, temporal analysis, an interactive dashboard, and cloud infrastructure.

Key accomplishments include:

  • Implementing real image and video inference with OpenCV 5 and YOLO11 segmentation.
  • Building an interactive dashboard with visual explanations and temporal metrics.
  • Implementing temporal confirmation and explicit handling of insufficient video data.
  • Successfully validating the application on AWS ECS/Fargate with Amazon S3 integration.
  • Reproducing the application from a fresh Git clone using Docker and the official model checkpoint.
  • Conducting an independent evaluation using 20 real videos: 10 flooded and 10 non-flooded.

Independent evaluation results:

  • Accuracy: 75.00%
  • Precision: 66.67%
  • Recall: 100.00%
  • F1 Score: 80.00%
  • True Positives: 10
  • True Negatives: 5
  • False Positives: 5
  • False Negatives: 0

These results are specific to our small experimental evaluation. The five false positives highlight the need for further improvements before any real-world operational use.

What we learned

This project helped us understand the challenges of developing computer vision systems for real-world environments.

We learned that successful model predictions on individual images do not necessarily guarantee reliable decisions across complete videos.

We also learned the importance of evaluating false positives, maintaining reproducible experiments, integrating AI models into complete applications, and documenting limitations transparently.

Working with OpenCV 5, YOLO, FastAPI, Docker, and AWS gave us practical experience connecting machine learning research with software engineering and cloud infrastructure.

Most importantly, we learned that an AI system intended for safety-relevant applications must prioritize reliability, transparency, and careful validation over impressive-looking predictions.

What's next for FloodLens AI

Our next steps include:

  • Expanding evaluation with more diverse and independent urban road scenes.
  • Improving discrimination between actual flooding, wet pavement, and reflective surfaces.
  • Studying camera-specific calibration and physical water-level estimation.
  • Exploring integration with weather information and physical water-level sensors.
  • Investigating future support for continuous camera monitoring.
  • Improving security, privacy protections, and deployment readiness.

FloodLens AI remains a research prototype and is not intended to replace official emergency monitoring or decision-making systems.

Our long-term vision is to contribute to more accessible, interpretable, and reliable computer vision tools for urban flood monitoring.

Built With

Share this project:

Updates

Submission history