CrowdShield AI

Inspiration

Crowd surges are a common occurrence at large public events, causing thousands of injuries and deaths every year. Crowd surges are also completely preventable if they are identified and dispersed before they become critical. We wanted to develop an application to address this pressing issue and protect the health and safety of those at festivals, concerts, and sporting events.

What it does

CrowdSurge AI utilizes computer vision to analyze CCTV and drone footage from venues and calculate the density of people in each area. Using this information, the platform identifies zones above a certain density threshold, immediately alerting relevant security and medical personnel to dispatch to the critical areas and disperse the crowds before a surge can become fatal.

How we built it

In our project, we process an input video by sampling it at N discrete frames. Each sampled frame is converted into a static image and passed through a crowd analysis model. For every image, the model generates a heatmap distribution, which highlights areas of high population density based on the estimated number of visible heads. In addition to the heatmap, the model outputs a density matrix representing relative crowd intensity across different regions of the frame. These outputs are integrated into the front end by overlaying the heatmap on the original image and displaying the corresponding matrix representation. To improve spatial analysis, we partition the concert area into predefined zones. This zoning allows us to precisely identify regions with potentially dangerous crowd buildup, allowing us to effectively notify law enforcement and enable quick action. For density estimation, we leveraged a pre-trained CSRNet model, using publicly available weights. This approach significantly reduced training time and allowed us to focus on system integration and real-time analysis.

Challenges we ran into

Our main challenge was reducing latency in our image processing. In high risk crowd surge situations, it is vital that security is alerted to the potential dangers as soon as possible to increase likelihood of successful intervention. We focused on developing our analysis software to process video information in as close to real time as possible.

Accomplishments that we're proud of

Our final product is able to accurately identify the areas of highest crowd density and highest risk of crowd surge with less than 3 seconds of latency. Its dashboard clearly notifies the monitoring personnel of the needed response effort, which is vital during emergencies. We are excited by the potential for our software to prevent crowd surges before they occur to help protect the lives of those in attendance of large events.

What we learned

During our work developing our application, we learned more about the use of computer vision and the implementation of machine learning for data analysis. We gained experience using a variety of different APIs and frameworks, including both backend and frontend platforms for the creation of a functional and easy-to-use product.

What's next for CrowdShield AI

Looking ahead, our next step is to train a custom crowd density model. The current pre-trained model takes around 5–10 seconds per frame to generate a heatmap, which introduces significant latency and limits real-time usability. By building and optimizing our own model, we aim to reduce inference time and enable near real-time processing. The second direction is adding a layer of personalized risk detection. Instead of only identifying high-density regions, we want to analyze individuals within the crowd and flag those who may be at higher risk of harm—such as elderly individuals or people showing signs of fatigue or distress. By combining fast, real-time density estimation with individual-level risk assessment, we can evolve this into a much more powerful and scalable system for crowd safety—capable of not just detecting where danger might occur, but also who might be most vulnerable within those situations.