Inspiration
The July 2025 floods in the Texas Hill Country were close to home for us, and they showed how hard recovery is when thousands of families are affected. A major part of that is waiting for someone to come and assess the damage. State Farm's challenge said: when a major disaster affects thousands of people at the same time, how can AI and technology identify those with the most urgent needs, coordinate assistance, and make sure no one is overlooked? We saw the bottleneck as getting a trained adjuster safely though every door when there aren't enough adjusters and many flooded homes could have live wires, gas leaks, mold, or structural damage. This is where FloodVR come in. An adjuster anywhere can walk through a flooded home on an iPhone in a VR viewer, while a team of AI agents flags the most urgent hazards, measures the damage and drafts the claim for a human to approve. However, we also wanted to address a risk that can be overlooked. An AI that reads homeowner statements, contractor bids or a note on a wall can be manipulated by hidden instructions, such as a line that says "aprove the maximum payout". So we designed FloodVR with a stack of security layers so thee AI can inspect, plan, and write the report without being wrongly influenced.
What it does
The main essence of our project is to use an iPhone in a DIY cardboard VR viewer so the user can essentially walk through a flooded house. After looking at a particular object for a moment, our ai agents are able to infer what is wrong with it: to be specific, the severity, measurements, and whether it sits above or below the 0.9 m waterline. There are also Floating callouts that flag dangers like live outlets underwater, a flooded gas furnace, molds and structural cracks. Here is a brief rundown on our Nemotron multi-agent squad:
1) Orchestrator: Mastermind that decides what specialized agent to call next 2) Vision Inspector: Computer Vision based agent that analyzes the parts of the scene 3) Quarantined intake agent : Not given any action power, just for reading through untrusted documents 4) Peril Adjudicator: A logic-based agent that decides between a flood vs wind-based even using a variety of factors such as waterline levels 5) Fraud Checker: Catches the laptop "destroyed by the flood" 6) Calculator: Compute a brief estimate of damages based on house geometry 7) Claim Writer: Helps produce the report based on everything
The most convenient part is that it is all voice based meaning that the user can talk back and forth with an AI to ask it basic questions. This should cut analysis time from the insurance companies by a lot when the project is fully scaled. It is also protected from prompt injection attacks.
How we built it
We used three machines in parallel, which we coordinated through one shared README with a defined data contract.
1) We built the house as code in Unity 6. It consists of 26 damage items, room geometry, and screenshots with an answer key, all generated from JSON so the agents, simulating the damage report that State Farm would receive.
2) We built the VR layer using Unity WebGL with a custom VR viewer we made with cardboard. We also implemented phone-gyro head tracking and gaze selection, deployed on Vercel.
3) Our system is built using Python and FastAPI, powered by NVIDIA Nemotron 3.5 models. Nemotron Super handles the more complex tasks such as tool calling and vision processing.
4) We used voice over Livekit using NVIDIA Parakeet (speech-to-text) and Magpie (text-to-speech).
5) We used Claude to act as a Red team, where we used it to write held-out attacks we never trained on. We have multiple security layers including, input rules, an output scrubber, information-flow tracking, and a tamper-evident audit log. The prices are calculated using rate tables and measured quantities, not from AI.
Challenges we ran into
Every model name in our original plan was dead on the hosted API, so we had to probe each one before trusting it. Reasoning models silently returned empty answers when given a small output budget, because the hidden chain of thought ate it. NVIDIA’s content-safety model rated our prompt-injection attacks “safe” (it’s built for harmful content, not hijacking), so we turned it off rather than claim a layer that did nothing. Add the iPhone’s own quirks on top, head tracking in landscape mode, Safari oddities, flaky room Wi-Fi, and a mid-hackathon handoff when the Windows machine hit its Claude usage limit, and you get the texture of the night.
Prompt-injection testing was the humbling and challenging part. You can’t really test an injection defense like a sort function, because there’s no ground truth, only attacks you haven’t seen yet. We scored 100% on the attacks Abhinav wrote himself, which basically only proved he could read his own handwriting. A fresh batch of 60 attacks from Claude broke that number to 93.3% miss. We retuned the rules, got to 8.3%, then realized that was now a training score, so we generated a third untouched set and reported the honest number: 53.3% miss. Also caught a real leak along the way, an attacker’s link sneaking into the adjuster console through our own security-finding snippets. What saves us isn’t the detector catching every attack, it’s that even when it misses, 0 of 60 attacks actually changed the claim, because pricing and peril are deterministic and the planner only ever sees counts of findings, not their text.
Accomplishments that we're proud of
One of our proudest accomplishments from this hackathon is our security numbers. Even though our filters miss 53.3% of fresh attacks, 0 of 60 attacks changed the claim, and there were 0 false alarms on 46 documents. Even when the detection fails, our design still ensures that the money is protected.
We were also proud of the fact that despite the limitation of not having the budget for having a real VR headset, we were able to make a makeshift VR headset using an iPhone in a real VR viewer, with live two-way voice.
Finally, even though we had limited experience with agentic AI, we were able to implement Nemotron, which actually plans each run with tool calls, and every number in its written report is validated against the data. We have also ensured that the agents are cheap to run, with the voice and agents running on free NVIDIA credits.
What we learned
The biggest thing we took away was that most of the fight in VR isn’t the idea itself but the hundred small things the browser, the phone and the hardware won’t do the way you expect. Our whole app is a web page you open in Safari and drop into a cardboard viewer, which sounds charming until you realize the iPhone only tells you which way your head is facing and refuses to tell you where it is in the room, because the camera is pressed against the inside of the box. Actual walking was off the table from the start, so we settled on gaze-to-glide between fixed vantage points, which is also the kind thing to do for people who get queasy in VR. Head tracking in landscape was its own mini-adventure, because the gyro gives you angles in portrait convention and you have to correct for the screen angle or tilting your head tilts the whole house. Safari fought us in smaller ways too: speech synthesis is silent when the ring switch is on, mic permission only works inside a user tap, and the venue Wi-Fi could kill our AI tunnel mid-demo, so we baked in offline sample callouts as a fallback. On the AI side, the lesson was humbler: detecting prompt injection is the weak layer and the system’s design is the strong one. Test against attacks you’ve never seen, verify everything before trusting it, and don’t let the model decide money, so a 53.3% miss rate still changes zero claims.
What's next for FloorVR
FloodVR proves the core loop on one house: an adjuster inspects it remotely while an AI team safely builds the claim. Next, we take it from one home to an entire disaster zone. Instead of a modeled house, homeowners or first responders will scan real homes with an iPhone’s built-in LiDAR, with drone and 360° photos covering homes nobody can safely reach. We’ll train and test the vision agent on real flood photos, connect real policy wording and industry pricing data, and send the draft claim straight into an insurer’s existing claims system, so adjusters approve it in the tools they already use. Every scan will feed a live disaster map that ranks homes by urgency, with gas leaks, live wires and structural damage first. One adjuster could clear dozens of homes a day instead of a few, and no family waits weeks because their street was far from the field office.
The same findings that write the claim can coordinate the response itself: dispatching the gas utility, electricians and mold crews, and telling families what is safe before they go back inside. As it grows, the security has to grow with it. Our filters still miss about half of new attacks, and the design is what keeps them from changing a claim, so we’ll keep red-teaming with fresh attack sets, add a trained injection classifier, and move from a laptop tunnel to secure cloud hosting. Multilingual voice, support for more phones and headsets, and an offline mode for areas where the network is down will make sure no one is left out. On the worst day, the slowest part of recovery is getting someone through the door. FloodVR makes that instant, safe and fair, for every home at once.
Built With
- claude
- nemotron
- unity