StabiliSense Engine

StabiliSense Engine

Comment

Inspiration

Falls are a classic “seconds matter” problem, especially for people with Parkinson's. Most systems detect impact after it occurs. We wanted to build something that could forecast instability before impact, reconstruct the biomechanics of the event, and provide a clean workflow for review and export.

StabiliSense Engine was built around that goal: early warning and interpretable kinematic analysis in a single system. It may not prevent falls directly, but it will help us understand them.

What It Does

StabiliSense Engine is a real-time and offline biomechanical instability analytics platform.

It can:

  • Run live from a webcam, stored video file, or ESP32-CAM WiFi stream
  • Continuously estimate pre-fall risk using a multi-model pipeline
  • Provide up to 0.8s lead time before confirmed fall behavior
  • Display a structured dashboard with risk gauge, timelines, biomechanical indicators, brace detection, and 3D pose views
  • Record WiFi camera sessions and normalize them for reliable browser playback
  • Extract and export the most recent fall clip with configurable pre/post context
  • Generate structured incident summaries using Gemini for kinematic interpretation

The system separates real-time monitoring from deep post-event analysis, allowing both fast prediction and detailed reconstruction.

Feature Overview

Live Monitor Mode

Designed for real-time instability forecasting.

  • Live fall risk meter with escalating warnings
  • Continuous center-of-mass tracking
  • Joint asymmetry and motion metrics
  • Arm bracing detection
  • Fall intensity estimation
  • Risk timeline visualization
  • Event logging with automatic clip capture

This mode prioritizes low-latency inference and predictive feedback.

Analysis Mode

Designed for detailed reconstruction and review.

  • Upload recorded files or load ESP32 captures
  • Smooth timeline scrubbing and replay
  • Joint angles vs time plots
  • Center-of-mass spatial and temporal trajectory
  • Pose reconstruction at instability inflection point
  • Symmetry analysis across limbs
  • Stand / sit / lie classification
  • Structured Gemini-generated summaries

This mode focuses on interpretability and biomechanical insight.

How We Built It

StabiliSense Engine is a modular Python system.

Pipeline

  • OpenCV for video ingestion
  • MediaPipe for pose extraction
  • Custom feature engineering including:
    • Center-of-mass estimation
    • Velocity and acceleration metrics
    • Joint angle trajectories
    • Rolling instability statistics
    • Limb asymmetry features

Models

  • Binary fall detector (gradient-boosting family, with faster alternatives available)
  • Multi-class fall-type classifier (none / chair / stand)
  • LSTM temporal predictor over sliding windows
  • Physics-informed severity scoring and brace logic

System Architecture

  • Thread-safe session state and event tracking
  • Context buffering for fall clip packaging
  • Dash + Flask UI with separate Live and Analysis tabs
  • Browser-safe video normalization and range-aware serving
  • Direct ESP32 WiFi stream ingest and recording

Challenges We Ran Into

  • MP4 codec/container inconsistencies across browsers (especially Firefox)
  • Managing asynchronous Dash callbacks without output conflicts
  • Maintaining UI responsiveness during real-time inference
  • Reducing false triggers during posture transitions and startup
  • Balancing inference latency with model fidelity on consumer hardware
  • Designing fall clip export logic that reliably captures correct pre/post buffers

Accomplishments We're Proud Of

  • An end-to-end workflow: capture → predict → reconstruct → explain → export
  • Real-time instability forecasting with meaningful lead time
  • Integrated ESP32 edge streaming and recording
  • Robust browser playback through normalization and range-aware serving
  • Smooth replay and scrubbing in analysis mode
  • Structured incident summaries grounded in kinematic signals

This feels like a system, not just a classifier demo.

What We Learned

  • Latency, buffering, and state management are as critical as model accuracy
  • Users need interpretable context, not just probabilities
  • Video serving details (codec, moov atom placement, byte-range handling) directly affect UX
  • Threading + web callbacks require disciplined boundaries
  • Demo reliability requires guard windows, state lockouts, and clear status indicators

What’s Next for StabiliSense Engine

  • User-specific calibration to reduce false positives
  • Better temporal fusion and uncertainty estimation across models
  • Lead-time distribution analysis and event-level evaluation metrics
  • Expanded fall taxonomy and rehabilitation metrics
  • Deployment hardening for edge devices
  • Structured reporting tools for clinical or research review
Share this project:

Updates