ShadowAthlete

Inspiration

Your only opponent is you.

Athletic progress is usually fragmented. A runner might track pace in one app, record tennis sessions on their phone, monitor recovery somewhere else, and have no meaningful way to understand how all of that contributes to them as an athlete.

We wanted to build something different: one athlete, one evolving digital twin, multiple sports.

Inspired by progression systems in games, ShadowAthlete turns real athletic sessions into a persistent representation of your performance. Instead of only comparing yourself with other athletes, your previous performance becomes the opponent you are trying to beat.

That became the idea behind Shadow You: Current You, Peak You, and Target You.

What it does

ShadowAthlete is an AI-powered multi-sport athlete digital twin platform built around Running, Tennis, Basketball, and Cricket fast bowling.

Athletes upload or record a training video. Our computer-vision pipeline extracts body pose information and sport-specific movement metrics. Those results feed into a shared athlete profile rather than remaining isolated inside each sport.

The platform is designed around three versions of the athlete:

Current You — your latest measured performance
Peak You — your strongest recorded performance
Target You — the version of yourself you are training toward

The experience combines video analysis, biomechanical metrics, athlete progression, training goals, recovery context, and a persistent digital identity.

Our interface lets athletes create their profile, select their sports and goals, capture or upload sessions, visualize pose tracking, review analysis, and progressively calibrate their Shadow.

How we built it

ShadowAthlete uses a modular architecture where individual sport analyzers feed standardized measurements into a shared Athlete Engine.

The backend is built with Python and FastAPI, with PostgreSQL and SQLAlchemy handling persistent athlete, session, and metric data.

For computer vision, we use tools including OpenCV, YOLO pose estimation, MediaPipe, and scikit-learn models, depending on the sport pipeline.

Running

Our running pipeline processes video with pose estimation and extracts biomechanical features including:

knee range of motion
hip range of motion
left/right movement symmetry
torso lean and stability
normalized ankle movement
stride timing

We trained a Random Forest gait classifier on a multi-subject markerless motion dataset. Instead of evaluating only with random frame splits, we used athlete-separated evaluation and leave-one-athlete-out cross-validation.

Our V2 gait classifier achieved 97.97% mean leave-one-athlete-out accuracy across 21 subjects when classifying slow walking, fast walking, and jogging.

The complete pipeline can take an MP4 video, detect the athlete's pose frame-by-frame, construct stride-level features, run the trained model, and return structured biomechanical results through the backend.

Tennis and the multi-sport system

Our tennis pipeline combines player/pose tracking with movement classification to identify tennis-specific actions and measurements.

The larger architecture allows Tennis, Running, Basketball, and Cricket to remain sport-specific at the analysis layer while contributing to the same athlete identity.

This was important to us: we did not want four disconnected sports apps. We wanted one athlete that learns from every sport they play.

Challenges we ran into

The hardest problem was turning an ambitious digital-twin concept into something technically credible within a hackathon.

Computer vision introduced challenges immediately. Camera angles, occlusion, multiple people, pose confidence, video quality, and differences between pose-estimation systems can all affect the measurements produced downstream.

We also discovered that high model confidence is not the same thing as high model accuracy.

That pushed us to evaluate models on athletes that were not present in their training data and to separate model classification confidence from biomechanical measurements.

Another challenge was creating a common athlete representation across very different sports. A tennis stroke, running stride, basketball movement, and fast-bowling action produce completely different raw measurements. We therefore designed the system so sport-specific analyzers can eventually map their results into shared athletic attributes instead of forcing every sport into the same model.

Accomplishments that we're proud of

We are especially proud that ShadowAthlete evolved beyond a UI concept into an end-to-end computer-vision system.

We built pipelines capable of going from:

Video → Pose Estimation → Movement Features → ML Analysis → Structured Metrics → Athlete Session

For running, we progressed from raw markerless pose data to stride extraction, feature engineering, model training, athlete-separated validation, and direct inference on MP4 video.

Our Running V2 gait model reached 97.97% mean leave-one-athlete-out accuracy across 21 athletes, giving us a much stronger evaluation than relying only on a single train/test split.

We also built a unified product experience around the technology: athlete onboarding, multi-sport selection, training goals, live capture concepts, pose visualization, session analysis, and the persistent Shadow identity.

Most importantly, every sport is being built around the same idea:

You aren't creating a new profile every time you change sports. You are developing the same athlete.

What we learned

We learned that the quality of the entire ML pipeline depends heavily on what happens before the classifier.

Reliable pose estimation, consistent landmark definitions, representative training data, useful feature engineering, and realistic evaluation all matter as much as the final model.

We also learned why testing on unseen athletes is important. A model can perform extremely well when similar athletes appear across training and testing data while struggling when introduced to someone completely new.

Building ShadowAthlete also taught us how to separate sport-specific intelligence from athlete-level intelligence.

Running should understand running. Tennis should understand tennis. But the Athlete Engine should understand the person behind both.

That architectural separation became one of the most important ideas behind the project.

What's next for ShadowAthlete

The next step is making the Shadow increasingly personal.

We want to move beyond activity classification toward deeper technique analysis, automatically compare new sessions against an athlete's previous best, and visualize exactly where Current You differs from Peak You.

We also want to improve cross-camera and cross-athlete robustness, expand the Basketball and Cricket analyzers, and further validate each sport pipeline on unseen athletes and real-world recordings.

Longer term, we want to integrate wearable data and recovery signals so ShadowAthlete understands not only how you performed, but also how ready you are to perform again.

The goal is for your digital twin to become a persistent athletic history — learning from every run, stroke, shot, and delivery.

Your biggest competition shouldn't disappear when the session ends.

It should evolve with you.

Train. Recover. Evolve. Your only opponent is you.