Big Brother is Watching You

(Always)

Comment 1

Inspiration

In a conversation with a biochemistry student, a study from Newcastle University was brought up. It showed that the feeling of being observed can significantly improve behaviour, even if it is an illusion. For example, when researchers used posters of eyes in a cafeteria, twice as many people cleaned up after themselves compared to the control group.

Humans are adept at detecting facial features, and this biological "gaze detection" likely evolved to quickly assess the presence of danger as well as for social reasons. This program serves to exploit this human behaviour.

We hope to help people work more efficiently; it might disincentivize people from impulsively opening their phones. It can even be deployed during an exam to discourage cheating. There's no need to record a user since they're primarily contained by a psychological barrier.

What it does

Our application is a cute anime girl with moving eyeballs that always stare at you when you are working or studying, pushing you to be productive. Essentially, it tracks your face in real time and follows you with its eyes. The window of the anime girl can be resized to fullscreen for those who really want to be stared down and have two or more monitors, and can also be resized to as small as just being noticeable in the corner of the screen for those who prefer to be stared down in the corner of their eyes or those who just work on one screen.

For best performance, check out our GitHub's teri-teri branch, which only tracks the first face detected when there are multiple faces. We have also implemented a feature that allows the program to tracks a given face no matter how many faces are detected through face recognition, which is located on our teri-teri-face-rec branch. Even though it doesn't have the best performance, we hope that it's at least shows our concept of urging the user to stay focused and productive despite the circumstances.

How we built it

We used Python's OpenCV to handle the live camera feed and displaying the edited image of an anime girl to the user. dlib's detection module was used to find the positions of each face in each frame, from which the eyes of the anime girl were moved towards the position of the first face detected. We also implemented a face detection feature that allows the program to only track onto your face (a reference image given at the beginning of the program) using dlib's sp and model modules to convert each face to a 128d vector then compare it to the 128d vector of your face, but after some testing, realized that its performance was not practical, so we opted to stick with only tracking the first face detected.

Challenges we ran into

  • Getting OpenCV to work requires a complex environment setup; some of our members struggled creating and working on a Python virtual environment
  • dlib face recognition is slow when the model parses the user's face frame by frame, so the practical solution that we opted to implemented was to only track the first face the model detects
  • Different time zones make it difficult for us to effectively communicate with each other, however, we kept updating information in our own discord channel to ensure all of us are updated.
  • A lot of small tweaks were being done behind the scenes such as reversing the face detection output when mapping it to the screen, then using the position of the face relative to the screen to calculate the movement of the eyeballs.

Accomplishments that we're proud of

  • Successfully detected the user's face and moving the eyeballs according to that movement
  • Managed to identify the user's face even if there are multiple faces popping up and tracking it

What we learned

  • OpenCV screen and image manipulation
  • dlib face detection and recognition
  • Problem-solving
  • Teamwork

What's next for Big Brother is Watching You

  • Instead of a static background image of an anime girl, which admittedly may not be very effective but we hope provides a proof of concept, having a deep fake look at you constantly would be immensely more effective. Imagine having your mom watching you work. You bet I wouldn't dare procrastinate.
  • Another idea was to connect the program with a vtuber. However, again, having a cute anime girl stare at you work may or may not greatly inhibit one's productive capabilities.
  • Although the ability to resize the application to just a corner of your screen is useful, creating a phone version of this app may make it more convenient since then it frees up the screen entirely.
  • Improve face recognition accuracy by building more complex models, and it will give users more options for different monitor modes. Building on this, implementing a fix that allows the face recognition feature to work smoother in real time would be high on our priority list, such as only running face recognition on 1 in every x frames to improve performance.

Built With

Share this project:

Updates