logo

LogiCare

Personalised care for your digital wellbeing. 2nd main prize at HackZurich'23.

Inspiration

Hackathon is a place where you can meet amazingly many people with unhealthy habits when it comes to digital devices. We believe there needs to be a way to preserve one's wellbeing even if they are a developer or designer. And this solution needs to be smart. Simple Pomodoro-like techniques don't quite cut it - they often interrupt a productive work cycle rather than work with your state.

What it does

Our application recognizes one's levels of stress and fatigue from input device usage patterns, as well as from webcam feed. It nudges users to take breaks when they need it: when they get tired and lose concentration. We believe that more data could be drawn also from keyboard usage, and nudges could also involve the input devices.

How we built it

We infer stress levels from mouse usage patterns, namely, from average speed and accuracy of mouse movements (Banholzer et al., 2021). Furthermore, we are looking at blinking patterns and yawning as signs of exhaustion (Ranti et al., 2020; Magliacano, 2023). Mostly using Python, we built a desktop application with data visualization, tracking, GUI and notifications for when users are no longer as effective with their time and can benefit from a break.

Challenges we ran into

Logitech SDK doesn't currently offer data about mouse movements or clicks, or about webcam feed, so we needed to use third-party libraries for that. Unfortunately, we did not have time to implement a way to e.g. snooze the nudges with a simple click of a special button on Logitech MX mouse, but that could be done. We did not have too much data, either - only one day of one person's mouse usage. A more thorough approach could include usage data collection for ML/DL analysis and pattern recognition.

Accomplishments that we're proud of

We used a face detection model, got facial landmarks, and calculated the eye aspect ratio to detect blinks. Using a similar approach, we calculated the mouth aspect ratio to detect yawns and combine the information as an indicator of fatigue. We also use information from mouse movement to calculate stress levels based on speed and accuracy, and built an app with a GUI which can nudge the user to take breaks via an unintrusive notification, using the gathered user data.

What we learned

It's great to have a diverse team with complementary skills and mutual support, but we should develop more in conjunction and plan more time for integration of components.

What's next for LogiCare

Our vision is that this technology could be integrated into Logitech software to enhance the value-in-use of Logitech devices, as well as utilize the company's market influence to promote healthy ways of working. Further work can be done to integrate current research on blinking and device usage as an indicator of stress, fatigue, and cognitive load.