It’s the question on every student’s and parent’s mind: will schools reopen? And if they do, will it be safe? Due to the COVID-19 pandemic, wearing masks has become a necessity to protect our community and families. It significantly reduces transmission rates, so it is mandated to wear these masks at school.

However, this is a significant adjustment, and will be difficult for younger students to correctly wear a mask on their own for many hours in the day . In addition, teachers are more susceptible to the virus than the kids, so if they need to fix masks for students, it puts them at risk. This should not be the case, and so I created the mobile application, MaskON to safety teach and enforce the use of masks at school. My project falls under the Health Track .

What it does

The app can be run on any mobile device and can be placed outside classrooms/halls/buildings. The student simply faces the app and the app will detect if the student is wearing the mask correctly . If they are, a “PERFECT” popup will occur. If the student is wearing it incorrectly, it will prompt a popup with instructions on how to fix it . The app can recognize when a person comes into focus and will automatically start analyzing the mask. This takes 2-3 seconds for the app to detect this.

Because this is all automatic, no touching is needed . This will prevent any possible transmission which sets it apart from other apps that require users to click buttons. In addition, the teacher can set an interval for how often to analyze to speed up the process. This makes it possible to analyze a whole class effortlessly.

How I built it

I used javascript and react-native to build the app. I also used the Expo IDE Client to test my application on my mobile device. The image recognition was aided by the Clarifai image recognition API , and I built a custom model to teach the computer to recognize proper mask wearing etiquette. I had to input json metadata for many example images that I used such as “MOVE MASK UP TO COVER NOSE COMPLETELY”, “PERFECT!”, and “TIGHTEN GAPS ON THE SIDES!” when I was training the model.

Challenges I ran into

Training the model to recognize these nuances was difficult and took hours of training before it got above the desired accuracy. But the most difficult challenge was making this all touch free . Initially, I was planning to use Google API Speech Recognition to prompt the app to begin analyzing when users spoke to the app. Students should not touch the device (if they did, it would defeat the whole purpose of reducing transmission at schools). But this took longer for the app to recognize and is not compatible for younger children who are still having trouble speaking clearly / are in a loud environment like a busy hallway. So I changed my idea . I instead set interval timers and used the Clarifai Api to call the function to start analyzing when it saw a child, person, or adult in focus. I found this to be much more efficient and compatible for all students and environments.

Accomplishments that I'm proud of

I wasn’t sure if the model was going to get accurate enough in time, but I was pleasantly surprised! It detects small nuances and mistakes people make when wearing masks like gaps near ears not not pulling the mask far up enough. In addition, I was happy that I was able to have the app analyze the student every time a student came into focus. This reduces transmission and speeds up the process.

What I learned

I am still pretty new to react-native, so I learned a lot about the functionalities through this project. I used more complex functions than I did in the past, but by troubleshooting, I not only came up with a functional application, but a stronger understanding of the language. I also researched more about UI to make this comprehensive for younger students to be able to use.

What's next for MaskON

In the future, this could be used in places other than schools that require masks (shopping centers, workplaces, gyms…). It is a universal application geared to all users . But, for the students, I would like to make more animations/sound effects to make the app more engaging and fun for younger students. After all, this is a big adjustment for them, so my goal is to keep the community safe while making sure that the kids get comfortable with wearing masks, too .

Also, I would like the app to send texts to the teacher informing him/her if any students entered without wearing the mask correctly. This streamlines the monitoring process and gives the teacher a record in case any contact tracing is required potentially. All in all, I envision this app successfully preventing transmission and protecting the communities nationwide to prevent another spike in cases this fall .

Share this project: