The main idea for this app arose in our minds, while we noticed that many households usually have one computer, which everyone uses commonly. These introduce a lot of problems like content restriction, age restriction, time control, privacy, and information protection (such as work-related applications). Overall built to improve the digital wellbeing of all generations alike, this app comes with multiple features including the parental lock.
What it does
➔ The app lets users add their faces, and lets the admins customize app preferences and allow screen time for each app for each user.
➔ The app detects the users accessing the system in real-time using facial recognition in real-time, and keeps a track of the time spent by each user on pre-decided applications, and is capable of handling a sudden change of user.
➔ If the parental lock is enabled for a user, Sentr3y3 continuously checks for any vulgar or profane apps/sites the user is visiting, and upon detection, kills the app. It also reports the incident to the admin (usually the parent) thus keeps the children safe from the dangers of the cyber world.
➔ In case the stipulated app time limit set for a user is going to be crossed, Sentr3y3 displays a warning message to the user to save his work. Upon reaching the said time limit, Sentr3y3 pauses the process that runs the application, thereby rendering the user unable to further use the application.
➔ A user-friendly GUI is in the development process and stands to be completed before the final evaluation.
➔ In case no user is detected, or an unknown user is detected, the app automatically locks the computer screen, adding an additional layer of security.
How we built it
We are keeping an eye on your digital well being. What we basically do is keep a track of the open processes that you ask us to monitor at the time of setup, we query user processes and monitor the usage of the whitelisted apps with all due permissions from the user. In the basic usage scenario where the user is a mature individual we just track the usage time and pause the apps once they reach a limit set by the user, the real twist comes in when you want to monitor the usage for your children. Firstly the admin panel has a face unlock feature so only you can modify access controls. In the scenario, the child tries to access some content that might have a negative impact on their psychology and natural upbringing, we instantly generate a notification and terminate the respective app. This way you can always ensure that your child stays protected and consumes digital content safely. Also, we feature an interactive dashboard that will help to view usage stats and keep a closer eye on how well you are doing, digitally.
The Current Application Flow
★ A Django backend server is started.
★ New users are added, their face encodings stored and their app preferences set by the admins.
★ The monitor script is executed and detects the users currently using the computer by taking snapshots in a regular interval. The script can detect multiple users at the same time, from the periodic snapshots
★ The monitor script also checks the apps opened by the user, and cross verifies it with predefined apps for that user. If any such app is opened, the monitor script updates the time spent on that app.
★ This script runs all these functionalities after a regular time interval (say 10 secs). If the script detects the presence of any profane apps in the user’s work area, it immediately terminates that app and displays a warning message.
★ If the allowable app usage limit is almost reached, the script displays a warning message to the user.
★ Upon exceeding the time limit, the script pauses that app, giving an option to the user to reset the time limit if they are an admin.
★ The use of Cron scheduler resets the daily and weekly usage statistics after the beginning of a new period (basically reset the time counters to 0).
Guido van Rossum, a computer with your favorite flavor of Linux, a webcam, and a human. On a serious note though, here are the pieces of tech we used to build our application.
Django Rest Framework A rapid quickfire framework to build web applications. Used to store the user and app information using its amazing ORM.
OpenCV, face_recognition library Used to detect faces in the periodic snapshots, store the encodings about the registered user faces for detection.
Python The omniscient language, encompassing everything, allowing rapid prototyping of ideas.
Cron Scheduler To daily reset the database about every app’s day to day usage, and to refresh the weekly statistics every week.
PyQt5, PyQt5Designer To make wonderful GUI applications, allowing us to monitor every user’s app usage, statistics, and registering new users, etc. using the QT Framework in python
XWindow Used to access the currently opened application windows in the system. It gives us all the window ids of the current applications in use
XProp For displaying window and font properties in an X server. Enabling us to get information about what content could be inside an application which is otherwise inaccessible from other userspace processes
Bash For access to necessary command-line tools such as grep, windows, xprop, etc.
Challenges we ran into, and how we solved them
We faced a lot of challenges !! One of the main issue was figuring out how to control an independent GUI application, which ran in a separate process, and get information about that application from an outside userspace process (without kernel permissions).
Then we came across Xwindow and we got to know a lot about the X Server System used by Linux to create Window based applications. Xwindow gave the window ids used by the application and also information about its dimensions, next we needed to get the process id to which this window belonged. Which we achieved by using another similar tool xprop which gave the properties of a window application. Finding these was a huge hassle but a great reward as we got to learn a lot.
The next biggest hurdle was age-restricted content used by children. To accomplish this, we got a great idea that using Xwindow, we can get the title name of the application. If an application shows age-restricted content, the title is bound to get on the radars of content restriction APIs. So we used Sight Engine's text moderation API to detect mature content when accessed by children, add reported the incident to the admin.
What we learned
We learned about how to pause, resume, stop, kill entire processes along with their children in python, giving us insights into Operating Systems. We learnt about the window system in Linux, and about the different tools provided by them, which were completely new and exciting for us to work with!!
What's next for Sentr3y3 Project
The next part of Sentr3y3 project is to include facial recognition based sudo permission granting, which is almost done, but couldn't be added due to time constraints.
Another thing we are considering is creating an immovable overlay over paused and timed out applications, to add more to the application experience
The application is designed mainly to be used in every household. We are planning to launch this application using a paid subscription model, with higher control over the content to be blocked using different tiers of the application subscription. Other features included would be cloud-based app information storage, remote app blocking, analysis tools for the admin or parents for better control of the computer, etc.