SteadiCam

Inspiration

SteadiCam started after I saw an older man whose hands were shaking struggle to eat. I did not know his diagnosis, but seeing how difficult such a simple everyday task had become made us think about how much motor symptoms can affect a person's independence.

That led us to Parkinson's disease, a progressive neurological condition affecting more than 10 million people worldwide. One major challenge for patients is that symptoms can change throughout the day as medication begins to wear off, while doctors may only see the patient during occasional appointments.

We wanted to build something that could help capture what happens between those visits using something patients already have: a webcam.

What it does

SteadiCam is a webcam based Parkinson's monitoring platform that tracks motor symptoms and connects them with medication timing.

Patients complete two short assessments:

• Resting tremor test to measure tremor frequency and movement amplitude
• Finger tapping test to measure tapping speed, rhythm irregularity, and movement decline

Patients can complete the assessment using the normal buttons, but we also wanted SteadiCam to remain usable when someone's hands are especially shaky.

If the patient simply raises their hand in front of the camera, SteadiCam detects it and automatically starts an ElevenLabs voice AI agent. The agent guides the patient through the assessment hands free, explains each test, and can help log medication information.

During the assessment, the voice agent also asks how the patient is feeling and adds a short note to the clinician's log in the patient's own words — so the dashboard isn't just numbers, it has context. Clinicians can add new patients and assign personalized medication and testing schedules, and the patient list automatically prioritizes anyone who's missed a dose, missed a test, or showing signs of wearing off.

The results are then shown on a clinician dashboard alongside medication schedules, actual medication events, missed tests, and adherence information. This helps clinicians identify possible patterns of symptoms returning as medication wears off.

How we built it

We built SteadiCam using HTML, CSS, JavaScript, Node.js, and Express.

For computer vision, we used MediaPipe HandLandmarker to track hand landmarks directly from the webcam.

For tremor analysis, we created a custom nonuniform Discrete Fourier Transform that uses the real timestamp of every webcam frame. This lets us analyze tremor frequencies even when webcam frames are dropped or delayed.

For the finger tapping test, we built a local peak prominence detector that measures:

• tapping rate
• movement amplitude
• amplitude decline
• rhythm irregularity

We integrated the ElevenLabs Conversational AI SDK directly into the patient workflow. Instead of adding a simple chatbot, the voice agent acts as part of the interface and can guide the user through the assessment and interact with SteadiCam through callable tools.

On the backend, we built APIs for patient data, assessment sessions, medication events, schedules, and adherence tracking.

Challenges we ran into

One of our biggest challenges was accurately detecting finger taps.

Our first version used a fixed threshold to count taps. The problem was that as the user's movement became smaller, later taps stopped crossing the threshold and were no longer detected. This was especially important because decreasing movement amplitude is one of the patterns we wanted to measure.

We rebuilt the system using a local peak prominence approach, allowing each tap to be evaluated relative to the surrounding movement instead of one global threshold.

We also originally estimated medication adherence by asking patients how many hours had passed since their last dose. After testing the dashboard, we realized this created confusing and unreliable data. We redesigned the system around actual medication events and doctor assigned schedules.

Finally, we originally planned to use Spring Boot for our backend, but development environment issues during the hackathon forced us to pivot to Express without losing the main functionality of the project.

Accomplishments that we're proud of

We are proud that SteadiCam goes beyond simply detecting a shaking hand.

We built a full pipeline that takes webcam movement data, converts it into meaningful motor measurements, connects those measurements with real medication timing, and presents the results in a way that a clinician can understand.

We are especially proud of the accessibility of the experience. A patient who is struggling to use a mouse can simply raise their hand and begin a voice guided assessment.

SteadiCam also requires no wearable or specialized hardware, making the system accessible from a normal laptop or computer with a webcam.

What we learned

The biggest thing we learned is that collecting health data is only useful when it has context.

A tremor measurement by itself tells us what happened at one moment. Connecting repeated measurements with when medication was scheduled, when it was actually taken, and how symptoms changed afterward provides a much more useful picture.

We also learned that accessibility should be part of the core design instead of something added later. Our ElevenLabs integration became more than a voice assistant. It became another way for patients to control and complete the assessment.

What's next for SteadiCam

The biggest next step is validating SteadiCam with real patient data and comparing our measurements against clinician administered assessments.

We also want to build personalized models that learn each patient's normal medication response over time and identify recurring wear off patterns.

Future versions could include:

• secure patient and clinician authentication
• encrypted health data storage
• EHR integration
• caregiver access
• mobile assessments
• automatic clinician alerts
• more advanced longitudinal symptom prediction

The long term goal is to help clinicians understand what happens between appointments, while making at home monitoring easier for patients whose motor symptoms can make traditional interfaces difficult.

Built With

• css3
• elevenlabs
• express.js
• html5
• javascript
• mediapipe
• node.js
• rest-api
• webrtc