BowlSafe AI

Agentic Pose Coach for Fast Bowling in cricket​

Comment

BowlSafe AI: Agentic Pose Coach for Fast Bowling in cricket​

TL;DR Turn any phone video into an injury-aware fast-bowling coach. We detect run-up and delivery-stride keypoints, compute joint angles, benchmark them against biomechanical ranges, and return Red/Amber/Green flags plus simple fixes.

Inspiration

We start with Devonie—a U.S. teen with Caribbean roots who learned fast bowling from YouTube. She built pace, but developed lower-back pain. Local labs and wearables were out of reach. We asked: Can we give Devonie lab-style feedback with just a phone clip? That question shaped the project.

What we built

Agentic Pose Coach—an end-to-end workflow that:

  1. Normalizes the input video for stable inference.
  2. Runs 2D pose estimation (HRNet) to extract COCO-17 keypoints per frame.
  3. Computes angles and metrics that matter for fast bowling (e.g., shoulder counter-rotation, trunk lean, knee angles, stride length).
  4. Benchmarks those metrics against a knowledge base to flag Safe / Watch / Risk.
  5. Outputs a coach-ready report with quick drills and cues.

Phases we tag: Run-up → Back-Foot Contact (BFC) → Front-Foot Contact (FFC) → Ball Release (BR) → Follow-through.

How we built it

1) Agentic workflow in Postman Flows

We used Postman Flows (think n8n-style blocks) to keep the demo deterministic and portable.

Blocks

  • Trigger: Start run with a video URL.
  • Normalize: Resize/crop to model-friendly dims (e.g., 384×288), standardize fps.
  • Pose: Call HRNet inference API; return COCO-17 keypoints per frame.
  • Angles: Transform keypoints → angles & phase metrics.
  • Benchmark: Compare against KB thresholds; produce flags.
  • Report: JSON/HTML card with metrics, flags, and 2 drills per flag.

  • If/Condition guards:

    • If frames > 2000 → decimate.
    • If avg keypoint visibility < 0.3 → retry normalization.
    • If fps > 60 or < 15 → resample for stable timing.

2) Pose → Angles (math)

Let (p_a, p_b, p_c \in \mathbb{R}^2) be joints (e.g., hip–knee–ankle). The joint angle at (a) is: [ \theta(a;b,c)=\cos^{-1}!\left( \frac{(p_b-p_a)\cdot (p_c-p_a)}{|p_b-p_a|;|p_c-p_a|} \right) ]

Derived bowling metrics

  • Shoulder counter-rotation (SCR), BFC→FFC Let shoulders (s_R,s_L), hips (h_R,h_L). [ \phi_s = \operatorname{atan2}\big((s_R-s_L)_y,(s_R-s_L)_x\big),\quad \phi_h = \operatorname{atan2}\big((h_R-h_L)y,(h_R-h_L)*x\big) ] [ \text{SCR} = \left|,\big(\phi_s-\phi_h\big){\text{FFC}} - \big(\phi_s-\phi_h\big)*{\text{BFC}},\right| ]

  • Trunk lateral flexion at/after FFC With mid-shoulder (m_s) and mid-hip (m_h): vector (v = m_s - m_h). [ \text{TrunkLean} = \left|,\tfrac{\pi}{2} - \operatorname{atan2}!\big(|v_y|,;|v_x|\big)\right| ]

  • Front-knee extension at BR (\theta_{\text{knee}} = \theta(\text{knee}; \text{hip}, \text{ankle})).

  • Stride length (scale-invariant) at FFC \text{stride}{\text{norm}} = \frac{|a{\text{front}} - a_{\text{back}}|} {|\text{hip}{\text{front}} - a{\text{front}}|}

  • Run-up consistency Track mid-hip displacement over time; smooth velocity with Savitzky–Golay: [ \tilde{x}_t = \mathrm{sgolay}(x_t,; \text{window},;\text{poly}) ]

Smoothing & phase picking

  • Temporal smoothing: Savitzky–Golay on coordinates and angles.
  • Events: BFC/FFC via peaks in vertical GRF proxies—approximated by sudden COM decel; practically, we detect rapid hip/ankle deceleration and front-ankle plant patterns.

3) Knowledge base → Flags

We keep thresholds configurable by age/history/role. Defaults (illustrative):

  • SCR (BFC→FFC): Green (<30^{\circ}), Amber (30!-!40^{\circ}), Red (>40^{\circ}).
  • Trunk lateral flexion @ FFC: Green (<15^{\circ}), Amber (15!-!25^{\circ}), Red (>25^{\circ}).
  • Front-knee @ BR: More extension often correlates with pace; we flag under-extension relative to the athlete’s baseline.
  • Stride norm: Personalized band from first three sessions; we flag instability, not absolute length.

The classifier is a simple rules engine: [ \text{flag}(m)= \begin{cases} \text{Red}, & m>u_r[2pt] \text{Amber}, & u_a < m \le u_r[2pt] \text{Green}, & m\le u_a \end{cases} ] with (u_a, u_r) pulled from the KB.

What we learned

  • Biomechanics ≠ black box. Simple, explainable 2D metrics can flag common risks (e.g., high SCR and trunk lean) and guide drills.
  • Agentic > manual. Postman Flows let us encode guardrails (fps, frame count, visibility) so the demo always completes.
  • Personalization matters. Relative change (to self) beats absolute targets for stride and approach consistency.
  • Design for coaches. Red/Amber/Green with 1–2 drills per flag reduced cognitive load vs. raw charts.

Challenges

  • 2D limits. Without depth, extreme camera angles hurt accuracy. We mitigated with scale-invariant ratios, consistency checks, and phase-specific heuristics.
  • Phase detection in the wild. Varied fps and motion blur made BFC/FFC tricky; we added velocity-based cues and fallback rules.
  • Occlusions. Arms overlap torso near BR. We used confidence-weighted angles and dropped frames below a visibility threshold.
  • Compute & robustness. Long clips ((>!2000) frames) and very high fps spiked runtime; decimation + early exits kept latency low.
  • Benchmarks curation. Literature varies; we exposed thresholds as KB entries so coaches can adapt them.

Tech stack

  • Inference: HRNet (COCO-17 keypoints) via REST API
  • Workflow: Postman Flows (Normalize → Pose → Angles → Benchmark → Report)
  • Angles/Phases: Python (NumPy), temporal smoothing (Savitzky–Golay)
  • Reporting: JSON + HTML card; CSV export for coaches
  • Storage: Session summaries with phase tags (BFC/FFC/BR)

Results (demo clips)

  • Detected SCR spikes between BFC→FFC and flagged trunk lean out of band.
  • Produced a one-page coaching card with Green/Amber/Red, plus two drills per flag.
  • Users could re-record and see deltas session-to-session.

What’s next

  • Dual-view capture (front + side) and monocular 3D lift for better trunk/hip estimates.
  • Auto-drill prescription based on which metric is Red and the athlete’s history.
  • Personalized baselines and progress graphs across weeks.
  • Mobile app capture with on-device inference acceleration.
  • Coach dashboard for teams/academies and white-label reports.

How to run (demo API sketch)

POST /normalize
Body: { "video_url": "...", "target": {"width":384,"height":288}, "fps": 30 }

POST /pose
Body: { "video_url": "<normalized_url>", "format": "coco17", "every_n": 2, "max_frames": 2000 }

POST /angles
Body: { "schema": "coco17", "frames": [ ...keypoints... ] }

POST /benchmark
Body: { "angles": {...}, "phaseEvents": {...}, "profile": {"age": 16, "role":"fast"} }

GET /report?id=<session_id>  # HTML card + CSV

Why it matters (WIIFY)

  • Athletes: Pace without pain. Clear, instant feedback from a phone.
  • Coaches: Objective, session-by-session metrics; less guesswork.
  • Judges: A working, explainable AI pipeline that puts biomechanics in everyone’s pocket.

Built With

Share this project:

Updates