-
-
Clean Caregiver View
-
Heatmap/Model View
-
Real-Time Audit Log
Inspiration
Millions of family and professional caregivers carry the heavy burden of taking care of the elderly. Those suffering from dementia often experience physical pain but their declining brain function makes it harder to recognise, process, or say that they are hurting. Consequently, caregivers at home face huge pressure trying to provide individual support whilst earning a living. As for care homes, they have to deal with underfunding and staff shortages, limiting the help and attention their patients need.
MyoTrace is designed as a non-invasive caregiver support tool that monitors facial indicators of pain without disrupting the patient's environment. The name comes from “Myo,” referring to muscle, and “Trace,” referring to movement tracking. These two elements reflect what our AI platform does: tracking facial indicators of pain to help caregivers recognise when non-verbal patients may be hurting.
What it does
MyoTrace turns an ordinary webcam into a silent pain detector. The patient just sits or rests in front of it as normal — nothing to wear, nothing to press, nothing touching them.
Every fraction of a second it reads the person's face and looks for three specific pain signals: the brows furrowing down, the eyes squeezing shut, and the upper lip tightening. First it spends a few seconds learning that person's normal, relaxed face, so it knows their baseline. It even tunes its own sensitivity to how much that face naturally moves at rest — so glasses or a resting squint don't cause false alarms.
If two or more of those three signals appear together (and the person isn't just smiling), it waits a moment to be sure it's real, then raises an alarm: a tone plays on the device, the moment is stamped into an exportable audit log, and — optionally — it can ping a nurse station over the local network.
Everything runs on the device itself. No cloud, no internet, no API keys — and no video ever leaves the room.
How we built it
Front-End (Streamlit): We utilised the open-source Python framework to rapidly prototype a reactive live dashboard — video feed, real-time signal bars, a confidence meter, a trend chart, and an audit log.
Back-End (Python + NumPy + OpenCV + MediaPipe): The engine is built in Python. Each video frame goes through Google's MediaPipe face model, which outputs 52 “blendshape” values — numbers describing facial movement. We map the pain-related ones (brow furrow, eye squint, upper-lip raise) into live percentages, smooth them over time, and feed them into a hysteresis classifier — the alarm switches on at one threshold and off at a lower one — so it doesn't flicker with noise.
Edge Architecture (Zero-API Inference): A self-contained model runs fully offline on the local CPU, with no cloud APIs and no internet needed. This keeps patient data on the device, allowing the tool to work, even without Wi-Fi.
Challenges we ran into
Python / MediaPipe Version Conflicts: Modern MediaPipe Windows libraries had a genuine bug where the native .dll was missing a free symbol, which crashed on the first face-detection call. We debugged the binding directly and wrote a small patch script (fix_mediapipe_windows.py) so anyone cloning the project can apply the fix in one command.
Front-End Limitations: With limited front-end experience, we leaned on Streamlit to prototype the dashboard quickly — but its reactive re-run model made a smooth live video feed surprisingly hard: the camera kept stuttering as the app re-rendered. We ended up streaming frames over a local MJPEG feed so the camera stays smooth while the AI runs in parallel.
Accomplishments that we're proud of
Working Edge-AI Prototype: An end-to-end tool that runs live facial tracking, baseline calibration, and multi-signal pain classification entirely on a local CPU — no GPU or cloud required.
Per-Patient Auto-Calibration: Instead of using one-size-fits-all thresholds, MyoTrace calibrates to each patient's resting face and adjusts its pain thresholds based on how much their face naturally moves. This reduces the need for manual tuning and helps the system adapt to differences between patients, including factors such as glasses.
What we learned
Working with Local Inference Models: We gained hands-on experience working with Google's MediaPipe face landmarker and its blendshape outputs (facial-muscle movement scores), learning how to build them into a real-time detection pipeline.
More Than Accuracy: We learned that building AI for caregiving is about more than detection accuracy. Privacy and trust matter too, especially when the system is monitoring someone’s face. That is why everything runs on-device, no video leaves the room, and the dashboard defaults to a simple caregiver view. The technical heatmap is available, but only when the caregiver chooses to view it.
What's next for MyoTrace
Native Cross-Platform Front-End: Move the dashboard out of Streamlit and rebuild it using a framework such as React or Tailwind CSS, giving caregivers more control and a more responsive interface.
Secure Multi-Patient Nursing Nodes: Extend the local network setup so a single caregiver dashboard can monitor multiple patient camera streams simultaneously across a secure intranet.
Fine-Tuning on Clinical Pain Data: Train a custom pain-expression model on real pain datasets (like the UNBC-McMaster Shoulder Pain Archive) so the detector can better distinguish pain-related expressions from normal facial movement.
