Gait Analyzer Arm

• 

  home screen

• 

  Signals

• 

  Cycles

• 

  AI explanation

• 

  performance increase vs regression

• 

  gemma throughput

• 

  gemma initialization and TTFT

• 

  pose throughput

• 

  pose median and p90 latency

I live with a rare disease which makes my bones very weak. I have undergone a major spine surgery so I have to monitor my spine health regularly, so I had built Gait Analyzer to analyze my Gait from just a side-vew walking video

That Gait Analyzer used MediaPipe on PC for pose estimation and Llama model using GPU for interpretation of the results. This setup turned out to be complex to setup and expensive to run.

So now I built Gait Analyzer Arm, where a single Arm optimized android app can do gait analysis using AI completely offline. I also built a benchmark script which factually proves that the app is a magnitude faster in pose estimation and LLM output when ARM optimizations are enabled.

Hackathon

This project was built for Track 3: Mobile AI of the Arm AI Optimization Challenge.

What it does

With just few seconds of the side-view video, the app analyzes the gait and explains it using a local LLM model.

More screenshots in the gallery.

XNNPACK A/B Benchmark

I have built a benchmarking script which extensively benchmarks the app with and without ARM optimizations (XNNPACK for MediaPipe and LiteRT).

The benchmarking script is included in the repository.

./benchmark

The benchmark script builds two identical versions of Gait Analyzer Arm app, one with XNNPACK enabled, another without XNNPACK and benchmarks them thoroughly.

These are the results from running the benchmark on a 5 year old android phone (OnePlus 9 pro):

Pose estimation median and p90 latency shows 26.47% improvement (lower is better)

Throughput in terms of FPS for pose estimation shows 37.83% improvement with XNNPACK (higher is better)

Even though gemma model initialization shows regression, time to first token shows 91.48% improvement (lower is better)

Gemma model shows 1024.95% prefill improvement and 1708.38% decode improvement with XNNPACK on a 5 year old phone (higher is better)

Optional Arm Streamline verification

./benchmark --with-streamline

Optionally, the benchmarking script can verify the use of XNN kernels.

Note: This requires Arm Streamline installed and might require a rooted android phone.

Built With

• android
• kotlin
• litert
• mediapipe