floodmark

Flood photos the model never trained on, and where it looked before calling them flooded. The heat is the model's evidence
A small portion of the 2,500 Atlanta camera views we sample, day and night
Floodmark's map of metro Atlanta. The red camera is the demo camera confirming a flood, with the frame it was judged on and the heatmap.

Inspiration

More than half of flood deaths in the US happen in vehicles. That number stuck with me. When a flash flood warning goes out for metro Atlanta, it covers whole counties, and a driver still has no way to know whether the underpass ahead of them is under water. However, Georgia already has over two thousand traffic cameras pointed at those exact roads. Nobody was watching them for water. Floodmark is our attempt to make the cameras watch themselves.

What it does

Floodmark checks 2,500 Georgia DOT traffic cameras around Atlanta every five minutes and marks each one on a map as dry, possible flooding, or flooded. When a camera is confirmed flooded, anyone subscribed to it gets an email. Clicking a camera shows the frame it was judged on and a heatmap of what the model looked at.

The detection is two small neural networks that run on a normal CPU in about $4\ \text{ms}$ per frame. Stage A gives the probability $p_A$ that there is water on the road. Stage B gives the probability $p_B$ that the water is a flood. The status of a single frame is

$$ \text{status} = \begin{cases} \text{dry} & p_A < t_A \ \text{flooded} & p_A \ge t_A \text{ and } p_B \ge t_B \ \text{possible flooding} & \text{otherwise} \end{cases} $$

with $t_A = 0.891$ and $t_B = 0.298$. A camera only turns red after $n$ flooded frames in a row ($n = 3$, or $2$ when it has rained nearby), and only when at least $1\ \text{mm}$ of rain fell at the camera in the last six hours. This is because one glare frame or one shiny bridge pier should never send an alert.

How we built it

First of all, the data. Nobody has labeled photos of flooded roads from these cameras, so we built a dataset of 9,337 road images from public sources: our own Atlanta camera captures, Iowa and New York DOT cameras, dashcam footage, and street photos from real floods. We wrote a small labeling tool and hand labeled 769 frames ourselves. We split everything by camera, location, and video so that no test camera ever appears in training.

Secondly, the model. Both stages are MobileNetV3, fine tuned with transfer learning and binary cross entropy loss. Flood photos do not look like grainy traffic cameras, so every training image gets a random mix of low resolution, sensor noise, text overlays, and night glare. We ran 26 training runs, tracked them in TensorBoard, and picked the winner on validation data only. Both stages export to ONNX with the Grad-CAM heatmap built into the graph, so serving needs no GPU and no TensorFlow.

Finally, the product. The backend is Python with ARQ workers on Redis. It pulls frames, runs the model, stores images in S3 compatible storage and results in Postgres, and sends alerts through Resend. The website is SvelteKit with a MapLibre map, deployed on Vercel.

Challenges we ran into

The biggest one was that the model kept calling a few dry cameras flooded, even with the three frames in a row rule. This is because a static camera shows the same scene every five minutes, so if the model makes a mistake once, it makes the same mistake three times and confirms itself. We fixed it by comparing each camera to its own normal instead of to a fixed threshold. A frame with flood score $s = p_A \cdot p_B$ only counts toward confirmation when

$$ s \ge \tilde{s}_{7d} + 0.10 $$

where $\tilde{s}_{7d}$ is that camera's median flood score over the last seven days. A scene that always scores 0.5 now needs 0.6 or more, three times in a row. We also skip frames that are byte identical to the previous capture, and we require rain nearby before any alert.

The second was that every Atlanta frame we could collect was a dry night frame. Daytime false alarms ran about seven times the night rate because of that ($1.4\%$ of daytime frames as opposed to $0.2\%$ at night). We are collecting daytime frames now, and the alerts the system suppresses are logged so they become the next round of training data.

We also found out that 511GA only refreshes a snapshot every few minutes, which put a floor on how fast we can confirm a flood. That was measured, not assumed.

Accomplishments that we're proud of

A teammate who did not build the model evaluated it once, on data never used for training or tuning. It caught $89\%$ of floods ($483/542$), up from $76\%$ for our first version. On the original test set, $126$ of $127$ flood calls were correct. On live Atlanta cameras it had never seen, in dry weather, it raised zero alerts after smoothing.

We are also proud of the honesty of the evaluation. Every weak spot in this writeup came out of it.

What we learned

Getting the model right was less than half the work. The rules around the model, meaning smoothing, baselines, the rain check, and the blocklist, are what turn a classifier into something a person can trust at 2 a.m. We also learned to test the thing we are actually shipping. Our first evaluation used a slightly different image loader than the server, and one camera flipped status because of it.

What's next for floodmark

It has never seen a real flooded Atlanta camera, because there has not been a flood since we started. The next storm is our real test set. After that, we want to retrain on daytime and rainy Atlanta frames, detect wet pavement as its own state, and offer confirmed camera reports back to forecasters as ground truth for their warnings.