Inspiration
Two of our teammates, Archna and Anirudh, had witnessed serious accidents at previous concerts and noticed how delayed the medical response was.
On discussing this with teammates, we were appalled to find out the number of deaths and injuries at concerts each year due to crowd surges and stampedes, which can be easily prevented.
What it does
Surge Protector uses drone technology to monitor crowded locations like music concerts and public protests.
Using a Dilated Convolutional Neural Network (CNN) model, we find the areas with the highest densities in the crowd, and alert organizers or authorities in a timely manner to prevent events such as stampedes. On Surge Protector's dashboard you can view these problematic regions with bounding rectangles and a density heat map, and with that information quickly send the necessary personnel to where they're needed most.
With statistics such as population in certain areas and time-series of population groups in the crowd, Surge Protector can be the key to a future of safer and more accessible concerts for all.
How we built it
Drone Infrastructure (provided by TreeHack Sponsors) - Use drone camera footage, with plans to add object detection and autonomous flying to navigate the venue, and streaming video footage and location data live time using a live-streaming API.
Backend: Uses Pytorch and CNNs for marking heads in a crowd Generate heatmaps using Matplotlib and OpenCV with custom thresholds. Used open sourced algorithms for training crowd counting model link
Wrote custom heatmap algorithms with Matplotlib and opencv to mark crowd, and temporal averaging for stabilizing bounding boxes generated by the model in Python3.
Frontend was designed in ReactJS with bootstrap.
Challenges we ran into
Our biggest issue was in the image processing side, trying to efficiently process video frames at a reasonable speed. Additionally, conversion between different intermediate formats such as numpy arrays, Python Images and Matplotlib plots was surprisingly painful. In one instance, converting a pyplot to a cv2 image without a margin became a technical issue which caused an immense amount of frustration .
Other problems would be incorporating the torch model, which was originally written in Python 2.7, and transforming it to Python 3.10 and running it in our Conda environment.
We also faced hardware issues while getting the drone feed, and even learning to fly a drone as none of our team members had used one before.
Accomplishments that we're proud of
We are incredibly grateful for TreeHacks for giving us the opportunity to meet amazing people and collaborate on ideas. As a team, we are proud that we met as diverse individuals from each of the four corners of the US, to hack on a project we felt passionate about.
We were also really happy that after almost 24 hours of continous hacking, we were able to get a real demo ready for our app, despite facing problems with conversions and cloud computing. All the pain and bugs we had to debug to get to the demo was worth it immediately after seeing the demo coming to light.
What we learned
We learned a lot about the potential of drone technology after speaking with some of the sponsors of the hackathon. We got to know about some incredible innovations happening in the autonomous aviation space and got to implement a tiny use case of the same for our project! Additionally, we had to research and use some interesting techniques for detecting the most dangerous areas of crowds, specifically keeping the bounded rectangles in consistent positions that made sense based on the density heat maps the the CNN produced.
What's next for Surge Protector
Surge Protector is just a proof of concept, and has a huge potential to grow as an project.
Fully autonomous drones from TreeHacks sponsors can be used to automate concert scans, and technologies such as infrared imagery and LIDAR can be used to ensure better safety of citizens by drones at night-time. Additionally, there is existing software which can fly towards a selected target, which we believe can help guide emergency personnel to those who need it most by shining a lot on dense spots in a crowd.
In the future, the Surge Protector dashboard can be expanded to include real time graphs and analytics in addition to a drone's live feed, allowing organizers to keep track of a constantly changing complex situations.
Log in or sign up for Devpost to join the conversation.