Inspiration

Imagine it's 2 a.m. and a house is on fire. A mother is on the second floor with her kids, and the stairs are the only way down. The first crew arrives within minutes, which is as fast as anyone could ask. They pull the hose, kick the door, and walk into black smoke where they can't see their own hands. They don't know if the stairs are on the left or the right. They don't know which room is the bedroom or whether there's a back door. They have to guess, and every second they spend guessing is a second that family is still inside.

We were talking about how many fires happen each day, and we realized that floor plans and building records are sitting in permit archives and county files. They're just locked away where no one on that truck can reach them. We're four freshmen who had never built anything like this, and we couldn't stop thinking about the people who run toward the fire and how they always walk in blind into a cloud of smoke and blazing flames, and the information that could help them is just out of reach. We wanted to put a better plan of action in their hands before they go in.

What it does

CloudScan turns a street address, or a photo of a house, into a 3D model of the building. A responder can spin it around, hide floors one at a time, pull it apart, and X-ray it so the exits and stairs light up. They can drop markers on it and plan a route in before they ever step through the door.

We also made a promise to ourselves early on. CloudScan would never pretend to know more than it does. Every value in the model is labeled as map data, read from a photo, or estimated, and a "How this was built" panel shows where each number came from. The footprint, size, and orientation of the house are real. The rooms inside are a typical layout until something measures the real interior. When someone is making a decision that could save a life, they deserve to know exactly what's solid and what's a guess. As more data get collected from more sources and APIs our system will run with a 100% accuracy.

Somewhere tonight, a crew is going to walk into a house they've never seen. We want to make sure that one day, they already know how to navigate in the house engulfed in smoke clouds before they even get there.

How we built it

It starts with an address. The US Census Geocoder finds it, with OpenStreetMap as a backup. If someone uploads a photo instead, we read the location from the image or the phone. Then we pull the building's real outline from OpenStreetMap, along with its height, roof shape, and which way it faces the street.

Next we fetch an aerial image, and the Gemini API looks at it, or at the user's photo, and tells us what it sees: how many floors, what kind of roof, which side the garage is on, whether there's a basement, and what the house is made of. Our layout engine takes all of that and builds the model. It keeps the true footprint, turns the front toward the street, lays out rooms, and keeps adding doors until every room can be reached. Exits are marked automatically.

Everything lives in a shared JSON format, and that choice mattered more than we expected. Our teammates building WiFi-beacon house mapping produce the same format, so their measured interior can replace our estimated one without changing anything else. It all runs in the browser on Next.js and Three.js, deployed on Vercel.

Challenges we ran into

The biggest wall was getting information at all. The records a firefighter would want, like real floor plans, permit files, and building details, are scattered across counties and cities, and almost none of it is available through an API. Some of it is on paper. Some of it sits behind a portal that was never built for a program to read. Some addresses had no building outline anywhere. It was frustrating to realize that the thing we wanted to build was blocked by how hard it is for the public to get at information that is technically public.

We couldn't wait for that to change, so we went looking for another way. We layered several free public sources so each one covered the gaps in the others. Gemini filled in what the maps were missing by reading photos and aerial images. And when nothing existed for an address, we used typical values and flagged them with a visible warning instead of hiding the gap. That's what made CloudScan work on any address and not only the lucky ones.

The other limit was our hardware and our APIs. We ran on free shared services with rate limits and the occasional outage, and we had no sensors to scan a real interior. That's why we can't honestly give you a perfectly accurate live demo. The interiors you see are an informed estimate. We would rather say that out loud than fake it, because a tool that firefighters might one day lean on can't be built on a trick.

Accomplishments that we're proud of

This was our first hackathon. We're four freshmen, and a day earlier none of us had built a tool like this. In about 24 hours we went from a typed address to a 3D building you can explore, and it runs on free public data with no special equipment.

But what we're proudest of isn't the 3D model. We are also proud of the fact that we are really transparent with our users of what information it is building these building layouts off of instead of making you assume, because someone's safety could depend on that difference.

What we learned

We learned that the hard part of a project that matters is rarely the code. It was getting the information, deciding what to trust, and being honest about the gaps. We learned that real data is messy and that you work with it instead of waiting for it to be clean.

We also learned how much four people can do when they split a problem and trust each other. We came in as classmates who had never shipped something together. We're leaving as a team that did, and that changed how we see what we can build. Our planning stage took much longer than we anticipated but it was fundamental into getting our scope of the project together and our plan of action for us to work best together as a team.

What's next for CloudScan

We think CloudScan can get better in two ways.

The base version should become more accurate by connecting more APIs, like parcel and county records, more imagery sources, and building databases. The goal is a model that gets closer to the real building using nothing but an address, so any department, even one with no budget, can use it.

CloudScan Pro is about the connection between the responder and the building. With WiFi-beacon mapping, CloudScan will detect the real interior as someone moves through it and replace our estimate with a measured plan. We want that connection and detection to be flawless, because that's the moment it stops being a good guess and becomes a map someone can trust with their life. We also want to experiment with getting this feature onto a phone so no additional hardware is required. After that come permit-archive import, dispatch integration, and an offline mode that works on the truck.

Additionally, we think CloudScan has real world practicality with more industries beyond firefighters, such as realtors or even the military. The information we can gather when scanning the houses about the interior plan and where all the important objects are located such as the pipes and electrical work.

WiFi-beacon mapping in action: https://vimeo.com/1244885010?share=copy&fl=sv&fe=ci

Built With

Share this project:

Updates

Submission history