-
-
Office Simulation
-
Before and After Comparison
-
Palace of Fine Arts
Inspiration
Crowd safety is often evaluated through static floor plans, occupancy limits, and assumptions about how people will move. But real venues are dynamic. Furniture gets rearranged, exits become congested, and the number of people in a space changes throughout an event.
We wanted to answer a simple question: What if event organizers could see how their venue would respond to a crowd before an emergency happens?
Traditional pedestrian simulation tools can be expensive, complex, and difficult to use without specialized knowledge. We envisioned Crowdy as a more accessible alternative—one that combines artificial intelligence, computer vision, and pedestrian simulation to help people understand how physical spaces behave under different conditions.
Our goal was to move beyond static floor plans and create a system that connects the physical world with an interactive simulation.
What it does
Crowdy is an AI-powered crowd monitoring and evacuation simulation platform that helps venue operators understand crowd movement, identify potential bottlenecks, and explore different evacuation scenarios.
Users can upload an image of a floor plan, and Crowdy uses AI to extract walls, doors, exits, rooms, and furniture. The resulting layout can be reviewed and edited in an interactive workspace.
Users can then configure occupancy, choose how people are distributed throughout the venue, block exits, and simulate evacuations. Crowdy visualizes individual pedestrian movement, congestion, exit utilization, and evacuation times. Users can also modify layouts and compare scenarios to understand how changes affect the simulated outcome.
Crowdy also connects to the real world through computer vision.
Users can upload photographs of a venue to estimate the number of visible people. For continuous monitoring, they can connect an iPhone camera using a QR code and stream sampled frames to Crowdy's crowd-estimation service.
The platform uses YOLO for individual person detection and a pretrained ZIP-N density-estimation model for crowded scenes. Camera observations estimate the number of visible people; Crowdy then generates starting positions within the floor plan and can run updated evacuation forecasts. These positions are simulated, rather than the tracked locations of people in the camera image.
For our Cal Hacks demonstration, we also created a reviewed simulation layout based on the Palace of Fine Arts, allowing us to demonstrate Crowdy using a venue directly connected to the hackathon.
Crowdy is designed for exploratory planning and decision support, not as a replacement for professional safety assessments.
How we built it
We built Crowdy around three major systems: floor-plan understanding, pedestrian simulation, and real-world crowd estimation.
The frontend uses Next.js, React, TypeScript, and Tailwind CSS to provide an interactive floor-plan editor, scenario configuration, simulation playback, and live analytics.
Our Python backend uses FastAPI and Shapely to process and validate building geometry. We integrated Gemini and Claude for AI-assisted floor-plan extraction, converting ordinary architectural images into editable geometry with real-world dimensions.
For evacuation modeling, we integrated JuPedSim, an open-source pedestrian simulation framework. Crowdy combines geometric path planning with agent-based pedestrian movement to simulate how people navigate around obstacles, interact with one another, and approach available exits. We also implemented congestion-aware exit selection and different population-placement modes.
For crowd estimation, we built a separate computer-vision microservice using Ultralytics YOLO and a pretrained ZIP-N crowd-density model. This allows Crowdy to handle both individually visible people and densely occupied scenes.
The iPhone camera connects through Safari using browser camera APIs and a temporary QR-code session. Images are processed by our inference service, and the resulting crowd estimates feed into the live analytics pipeline.
We deployed the frontend using Vercel, hosted our Python services on Azure Container Apps, and used Azure Blob Storage for persistent workspace data. We also integrated Auth0 for authentication and Terraform for infrastructure management.
Challenges we ran into
One of our biggest challenges was turning ordinary floor-plan images into geometrically valid simulation environments.
AI models can recognize walls and doors, but small errors in their positions can produce disconnected rooms, blocked passages, or unrealistic routes. We had to develop geometry validation, scale calibration, and editing tools to ensure that extracted layouts could actually be simulated.
Another major challenge was accurately estimating crowds from images. YOLO performs individual person detection, but densely packed crowds introduce significant occlusion and make people difficult to distinguish. We explored specialized density estimation to handle scenes containing thousands of people, while recognizing that accuracy still varies considerably across camera angles, image quality, and crowd conditions.
Scaling pedestrian simulation was another difficult engineering problem. Simulating thousands of individual agents requires substantially more computation than simulating a small room. We had to address performance, asynchronous execution, response sizes, and incomplete simulations without misrepresenting their results.
Finally, integrating the iPhone camera, computer-vision service, cloud infrastructure, and simulation backend into one responsive application required careful coordination across multiple independently deployed components.
Accomplishments that we're proud of
We're particularly proud that Crowdy evolved from an idea into an integrated system combining AI, computer vision, computational geometry, and agent-based simulation.
We built a workflow that begins with an ordinary floor-plan image and produces an editable, physically scaled environment capable of simulating individual pedestrians.
We also implemented a live iPhone camera integration that connects real-world crowd observations to the simulation workspace. Rather than stopping at a detected headcount, Crowdy can use updated crowd estimates to generate new evacuation forecasts.
Another milestone was creating a reviewed demonstration layout for the Palace of Fine Arts, including 15 mapped exits, and successfully running the integrated crowd-estimation and evacuation pipeline.
We're also proud of the engineering behind the interface: reproducible scenarios, geometry validation, cloud-persisted workspaces, independent inference services, and extensive automated testing.
Most importantly, we built something that demonstrates how simulation and real-world sensing can work together to support better-informed decisions about physical spaces.
What we learned
This project taught us that creating a convincing simulation is very different from creating a trustworthy one.
A visually realistic animation does not necessarily mean the underlying assumptions are accurate. We learned the importance of geometry validation, physical scale, reproducible experiments, and clearly distinguishing observed information from modeled behavior.
We also discovered that different computer-vision problems require different approaches. Detecting individuals in a small group is fundamentally different from estimating thousands of people in a dense crowd. No single model performs equally well in every setting.
Building live analytics taught us how to connect computer vision with computational simulation while managing latency, limited resources, asynchronous jobs, and changing inputs.
Above all, we learned how powerful the combination of AI perception and physics-based simulation can be. AI can help interpret the physical world, while simulation can help explore what might happen under different conditions.
What's next for Crowdy
Our long-term vision is to turn Crowdy into a platform for real-time, simulation-assisted venue operations.
We want to expand beyond estimating crowd size to understanding where people are located throughout a venue. This includes mapping camera observations onto floor plans, supporting multiple cameras, and tracking entrance and exit flows to produce better occupancy estimates.
We also plan to use computer vision to detect changes in the physical environment, such as newly positioned furniture, temporary barriers, or obstructed pathways, and incorporate those observations into the simulation.
Another major direction is simulation-driven layout optimization. Instead of only showing bottlenecks, Crowdy could propose alternative furniture arrangements or routes, simulate them, and explain the resulting trade-offs.
We also want to improve high-density simulation performance, validate our models against real-world data, and eventually support more complex, multi-floor venues.
Our ultimate vision is for Crowdy to become a bridge between the physical world and predictive simulation—helping event organizers move from reacting to crowd conditions toward understanding and planning for them.
Log in or sign up for Devpost to join the conversation.