DONT (DO Not Touch your face)

Action recognition result
GUI form
EXE file is available
Binary confusion matrix

Inspiration

To overcome pandemic of SARS-CoV-2 in the world, there are several basic protective measures against COVID-19 to wash your hands frequently, avoid touching eyes, nose and mouth, practice respiratory hygiene, and maintain social distancing spread according to the WHO advice(https://www.who.int/emergencies/diseases/novel-coronavirus-2019/advice-for-public).

Especially, it is very important to avoid touching eyes, nose and mouth and wash your hands frequently. However, it's very difficult to wash your hand perfectly (https://www.youtube.com/watch?v=4O0zkBQTgvI), and people unconsciously touch their faces more than 3,000 times a day. Therefore, we want to warn the touching their face by developing an automated monitoring software with deep learning.

What it does

This program can reduce the frequency of touching your face by giving you an alarm when you touch your face unconsciously with web-camera and monitoring with deep learning technology.

How we built it

Firstly, The action recognition neural network (I3D) was used to recognize the video clips on the action of touching the face (ver.I3D).
Secondly, for light version of deep learning model, MobileNet3 was used to recognize the action of touching the face with three consequent snapshot images (ver.MobileNet).
It was created in the form of EXE program so that it could be easily used by ordinary users.
We made and publish a dataset to be used freely to develop recognition software on the action of touching the face.

Challenges we ran into

More data are needed for higher and robust accuracy in various kinds of situations.
Lighter, faster neural network model is needed to be used in smartphone and/or ordinary PC without specialized GPU, etc.

Accomplishments that we're proud of

The first model can recognize face-touching actions in 0.08 sec with a GTX960 or higher GPUs (ver.I3D, 92% accuracy)
The second model can recognize face-touching actions in 0.07 sec with Intel(R) Core i7-6700 CPU 3.40GHz or higher (ver.MobileNet, 92% accuracy)
We have released out pre-trained action recognition model, which can be used freely

What we learned

We realized that people touch their faces unconsciously.
Fully automated monitoring with deep learning for detecting touch-the-face could be used to prevent SARS-CoV-2 from spreading.
It is very important to collaborate with teammates quickly.

What's next for DONT

To add sound alarm.
To add a "report" function based on 24-hour monitoring mode.
To modify DONT to Apply in CCTV, CPU-only machine and mobile phones in actual environment