Problem Statement

Older adults with dementia often experience loss of independence, distress, and aggression, increasing care demands and creating challenges for long-term care centres. These challenges, including staff burnout and limited mental health support, make it difficult to provide consistent care. Sensory therapies have shown promise in improving the well-being of older adults. Still, they are often operationally challenging due to safety restrictions, high workloads, and the need for specialized training.

Solution Overview

We developed a sensory therapy cueing tool to assist healthcare staff and older adults with dementia. This tool uses visual and audio cues to enable patients to complete sensory activities with minimal guidance. It also reduces staff workload by allowing supervision without physical interaction and enhances therapy outcomes.

Libraries Used:

Body tracking: MediaPipe and OpenCV

Visual skeleton: PyGame

Video capture: OpenCV

Technical Overview

The tool uses a webcam for body tracking and projects a scaled skeleton onto the patient's body. A green rectangle and spoken words guide patients on where to tap, and the patient will follow these cues in a pre-determined sequence. The touch detection algorithm registers a touch when the distances between the hand landmarks and body landmarks fall below a threshold. A sliding window approach with touch history tracking prevents false positives from momentary proximity.

Future Development

Plans include integrating machine learning (TensorFlow/PyTorch) for improved detection, predictive analytics, and personalized care. To improve scalability and care delivery, a digital management system can be used for tracking patient data.

Built With

• mediapipe
• opencv
• pygame
• python