Flock

Inspiration

During the ten year period from 1992 to 2002, 66, 787 people suffered significant injury and 232 people died at 306 outdoor music concerts around the world. In light of these concerning statistics, it becomes evident that effective crowd management and safety measures are crucial for ensuring the well-being of attendees at large-scale events. This is where Flock steps in.

What it does

Our web dashboard provides venue staff with a centralized hub for overseeing crowd dynamics and managing emergency responses. It is primarily focused on processing drone footage to enhance crowd management. Key features include: Live Feeds: Staff can view multiple live feeds from different camera angles capturing the audience, with a primary focus on drone footage. Crowd Density Detection: The system analyzes drone footage to detect the number of people in the crowd. It calculates the crowd density across various areas of the venue based on its size, providing a clear picture of crowd distribution. Crowd Density Heatmap: A real-time heatmap visualizes crowd density, helping staff quickly assess areas of high concentration and manage crowd flow effectively. Alerts: The system sends alerts when the human density exceeds safe thresholds, enabling prompt action by on-site security personnel to address potential crowding issues.

How we built it

Frontend: Next.js: We used Next.js to develop the frontend of our project, creating a seamless user interface for our dashboard. Next.js efficiently handles all backend requests, ensuring smooth and responsive interaction for users. For our authentication, we implemented Auth0, which is integrated with MongoDB to manage user accounts. This setup secures access to the web dashboard, ensuring that only authorized users can view and interact with the data. Backend: The backend is powered by Flask, which manages API endpoints and handles data processing for our video and image inputs. One key feature of our backend is our calculation design for crowd density. We integrate tensorflow, YOLO, and open-cv to map birds-eye view footage with accurate crowd density estimations, which are crucial for life saving analysis and risk assessment. With our database, it works in parallel with Auth0 to manage our database to storing and managing venue data, crowd analytics, and user information. MongoDB's flexibility and scalability make it an ideal choice for handling the diverse and extensive data involved in our project.

Challenges we ran into

Training a Custom Model: Initially, we planned to train a custom crowd counting model but encountered speed issues due to insufficient GPU resources. This challenge hindered our ability to use the model effectively in time. As a result, we pivoted to a fallback solution, which allowed us to meet our project goals. Image Processing Requests: We also faced challenges with sending image requests from the frontend to the Flask backend for processing our API. Our achilles heel in this project is connecting nextjs frontend API requests with backend responses. This includes handling file uploads and ensuring that image data was correctly transmitted and processed.

Accomplishments that we're proud of

Training a YOLO Image Detection Model: We had the opportunity to experiment with training a YOLO (You Only Look Once) image detection model. Although we had to scrap this approach due to time constraints, it was a valuable learning experience that gave us insights into object detection model training processes.
Crowd Density Calculations: We conducted our own crowd density calculations from scratch. This involved extensive research, including reading academic papers on crowd density limits and human tolerance. Developing this capability in-house was a significant achievement, as it allowed us to tailor the density calculations specifically to our project's requirements.
Creating a Custom Heatmap: We designed and implemented our own heatmap largely from scratch. This heatmap visualizes crowd density and provides critical insights into crowd distribution. Building this feature independently was a major milestone, showcasing our ability to develop complex visualization tools tailored to our needs.

What we learned

We learned that if something can go wrong, it often will. At every step of the way in developing Flock, we were presented with a new unforeseen challenge. It was good practice in keeping our patience and learning when to be resilient VS pivot to something new.

What's next for Flock

Testing with Drone Footage: We plan to test Flock with actual drone footage to evaluate its performance in dynamic and large-scale environments. This will help us understand how well our system adapts to different types of video inputs and enhances its accuracy.
Improving Crowd Detection in Low Lighting: We are focusing on enhancing crowd detection capabilities for concerts and events with lower lighting conditions. This involves refining our algorithms and integrating advanced image processing techniques to ensure reliable crowd monitoring even in challenging lighting scenarios.
Adding Specialized Metrics: We aim to incorporate additional specialized metrics, such as the speed of movement within different areas of the crowd. This enhancement will provide a deeper understanding of crowd dynamics and enable more effective crowd management by tracking whether areas of the crowd are facing increased pressure.