CrowdAI

Offers real-time streaming analytics and visualization tools to help users and business or government entities to make more informed, safer, and smarter decisions

Comment

Side Prizes

  • Most Interdisciplinary Prize
  • Best Overall Hack by Women in Tech

How Our Idea Came About

Because the team had a common interest in working on a project involving machine learning, Anita asked if machine learning could have helped with the investigation from the Boston Marathon bombings back in 2013, an event that shook up not only her hometown but also the world. Namely, investigators requested for any and all CCTV tapes near where the bombings occurred in order to help determine who might've placed and set off the bombs as well as piece together the timeline of events.

As someone who works at the vanguard of fields like network engineering and systems design and is keenly aware of the importance of addressing and protecting against constant dangers inherent in the world of high-speed global connections, Ishaan took a step back to see the big picture and added his perspective that at the heart of what Anita was talking about is extracting information from a video or image source, which lends itself to crowd analytics.

Haohui agreed that there is value in using artificial intelligence to transform real-time image and video data into safety insights. She saw the opportunity to apply video analytics to the current COVID-19 pandemic by helping determine how crowded a given location is, whether people are obeying social distancing guidelines and wearing masks. She took on the challenge of implementing real-time object detection, tracking, and distance calculation in under 24 hours, along with creating a front-end to enable the visualization of real-time annotated videos.

Project Considerations Discussed

Ishaan discussed two types of end users who would use this tool, and both Haohui and Anita elaborated on different use cases:

  • Individuals who want to know how crowded an area is before traveling there or whether there might be an event of interest happening (e.g. news reporter interested in https://www.bizjournals.com/boston/news/2020/06/08/black-professionals-stand-silent-at-faneuil-hall.html) taking place
  • Business and/or government entities who want to understand movement patterns (e.g. what spaces are not being engaged and why? for urban planning), assess safety in their jurisdiction and prepare to call for help or alert police of potential altercations (e.g. are people with knives and guns hanging out in storefront?), or even maximize profits (e.g. do customers linger longer frequent aisle more when moving item X near item Y or from reconfiguring shop layout?).

Our Approach to Offer Users Crowd Intelligence Data - What it does

In a nutshell, our app offers real-time video streaming analytics and visualization tools to help users and business or government entities to make more informed, safer, and smarter decisions. With our website, people can:

  • monitor area crowding, crowd flow, and venue capacity by numbers (this is geared towards business or government entities that own the relevant video footage. Due to privacy concerns, this footage will not be available to the public and is strictly meant to monitor the crowd in real-time.)
  • quickly assess area safety - whether users can practice safe social distancing (available to both the public and organizations alike)
  • anticipate potential threats - whether other individuals in the area are wearing masks (available to both the public and organizations alike)

Our app makes AI data-driven decisionmaking possible.

How we built it

Web app to introduce project and interface with Machine Learning Backend: Anita built a website using Angular.js for the first time.

Social Distancing Detection: Using the YOLO-v3 model, Haohui coded a human recognition model to calculate the distance between human beings and determine if they were abiding by safe distancing rules of being more than 2m apart from each other.

Face Mask Detection: Haohui trained a custom YOLO model from scratch to detect faces and determine if the person in the live videostream is wearing a mask properly.

With these two YOLO models, Haohui used OpenCV to annotate the videostream and coded the front-end with the Streamlit library to display the annotated video containing bounding boxes in real-time. With the crowd information gathered from the videostream, we then determine how safe the location is for others to visit. If there were many people violating social distancing guidelines or the location is too crowded, the user will be informed about the crowd levels and can thus make appropriate changes to their plans. This information is displayed as a dashboard for the consumer, coded using Streamlit and the Altair visualization library. With an interactive map, members of the community can explore crowd trends at different timings to best plan their trip to avoid large crowds.

Challenges we ran into

We tried recruiting front end developers on Slack and via Devpost with no such luck, so we had to figure out how to create a front end with our best design intuition. Midway through the hackathon, one of our team members lost wifi. As such, we were down to two members only, did not have access to the work done by him, and had to race against time to get the machine learning backend and frontend up within the remaining time!

Most Interdisciplinary Focus

  • Haohui Liu, Undeclared Major
  • Anita Yip, Computer Science Major. Also has degrees from 10+ years ago in Corporate and Organizational Communication (MS), and Environmental Studies & Media Arts and Sciences (BA).

Please know that we reference Ishaan Jaffer, who was with us at the beginning of the hackathon and then lost wifi and hasn't been in touch since. He worked on the cloud back-end with Azure, but unfortunately did not share with us his work. As such, there were no code contributions to this project from Ishaan.

In short:

  • Anita brought up a use case scenario from her hometown that technology can address, thought critically about what type of info can be made available and how that info might be presented appropriately to the public/consumers from her moral philosophy class, and distilled the essence of the project in words with her communications background
  • Ishaan steered the idea in the direction of cyber-physical systems and potential predictive analytics and walked the team through a methodical approach to design thinking and creating a project with social impact with his ECE background; proposed to leverage Microsoft Azure's solutions and features to analyze crowd intelligence and detect anomalies
  • Haohui, the resident machine learning enthusiast on the team determined the scope of hackathon project, and tackled the challenge of implementing multiple complex models in under 24 hours (even training a YOLO model from scratch), analyzing crowd analytics using OpenCV and displaying the information in real-time using Streamlit.

Crowd intelligence itself is interdisciplinary, as it integrates into all kinds of daily life applications such as real-time traffic monitoring, logistics management, urban planning, location-based security and more. Detecting anomalies, for example, requires thinking outside the box — not only in defining what is considered an anomaly (e.g. presence of a gun on site) but also in implementing algorithms to identify anomalies (e.g. people's behavior - eye movements identifying location of CCTV cameras in an area could be a precursor to crime happening in the coming days/weeks).

Accomplishments that we're proud of

All of us went beyond our comfort zones!

  • created our first angular.js website
  • implemented complex algorithms in a very short timeframe
  • worked to identify and detect anomalies

What we learned

A whole lot — especially with adaptability, going beyond our comfort zone, and making it work. Our team may be small, but we have ambitious ideas and the confidence to figure things out to make it happen.

Also, remember to share permissions to your repos and cloud projects at the beginning of the hackathon in case contingencies like wifi not available happening! This is Murphy's Law at its best for a hackathon— of course, that doesn't stop us from making it work.

What's next for Crowd Intelligence Assistant

  • More data analysis for the various use cases
  • Granted no ethical violations, save aggregated data for further analysis of trends and offer predictive analytics

Built With

Share this project:

Updates