Due to the pandemic, classes changed from in-person to online, hosted commonly by the video communication application Zoom. One day, in a typical zoom class in biology, a random participant joined and started playing very extreme, inappropriate content that frustrated the teachers and scared the students. There were a frequent amount of these occurrences. Later, we found out that this is known as zoombombing, which refers “to the unwanted, disruptive intrusion, generally by Internet trolls, into a video-conference call. In a typical Zoombombing incident, a teleconferencing session is hijacked by the insertion of material that is lewd, obscene, racist, homophobic, or antisemitic in nature, typically resulting in the shutdown of the session.” Thus, as victims of this problem, we wanted to find a solution.
What it does
Our application, ZoomSecure, aims to prevent zoombombing from occurring in all online classrooms. Through our application, the teacher would provide a class list, which typically includes all the names and pictures of students. Then, through facial recognition, when students enter the zoom meeting our application compares a short video preview of the student to a pre-existing student image list. This verification will help ensure that this student is part of the class itself, preventing all further zoombombings.
How we built it
We built this project through OpenCV and 2 different machine learning algorithms, the haar cascade algorithm and the LBPH algorithm. The primary language used was Python, and both algorithms were from the OpenCV library. First, we used Python to parse through the data used in our demo to make it easier to use for our machine learning algorithm. Then, we used the LBPH algorithm to train this data. The LBPH algorithm is a simple yet very efficient texture operator which labels the pixels of an image by thresholding the neighborhood of each pixel and considers the result as a binary number. With this trained data, we could now do facial recognition given a student image through a webcam! Now, we needed to detect the face within the webcam. For this, we used the haar cascade algorithm, an Object Detection Algorithm used to identify faces in an image or a real time video. Using pre-trained data available in the OpenCV library, we were able to detect our faces on a webcam screen. Combining these two algorithms together, we could now detect a person’s face from a webcam, and verify to see if they were part of the classroom.
Challenges we ran into
Although we knew that we should use the OpenCV library for facial recognition, we had no experience using this library. Further, we had a hard time trying to figure out what machine learning algorithm we should use. We also ran into multiple errors and had to debug it as we were learning how to use and manipulate these complex algorithms that we used in our project. However, after extensive research, we were able to understand how to use these tools.
Accomplishments that we're proud of
We are proud of creating an application that could have a real life impact on many people. In creating this application we had to learn a lot of new skills including many new types of algorithms including OpenCV, Deep Learning Haar Cascade Algorithm and the LBPH Algorithm.
What we learned
We did not have any experience with machine learning, using OpenCV and any machine learning algorithms for facial recognition prior to this endeavor.
What's next for ZoomSecure
We would like to create both a front end and back end website with a database for storing names, images, that serves as a predestination for entering a zoom meeting.