Inspiration
Professional tennis generates 1.88 million charted points — yet not a single open expected-value model exists. Every serious analytics framework in the sport (Hawk-Eye, IBM SlamTracker, Kovalchik's proprietary EPV) is locked behind paywalls or grand-slam NDAs. Meanwhile, football has xG, basketball has Expected Possession Value, and soccer has passing models — all open, all reproducible.
We asked: Why does tennis — a sport of isolated, discrete, perfectly observable decisions — lack the most basic analytical infrastructure every other sport takes for granted?
The answer wasn't technical. It was institutional. Tracking data is proprietary. Charting data is crowdsourced and messy. Nobody wanted to build on volunteer-contributed point logs.
So we did. We took Jeff Sackmann's Match Charting Project — 1,877,757 points across 3,000+ matches, freely available under CC BY-NC-SA 4.0 — and asked a question no one else had: If you could model the expected value of every single state in professional tennis using only publicly charted data, what would the optimal serve strategy look like — and how many points are players leaving on the table by ignoring it?
That question became Aim Gap.
What it does
Aim Gap is the first open, shot-level Expected Point Value engine for professional tennis — and the first to close the loop from raw data to causal, counterfactual, interactive tactical intelligence.
The system computes four layered metrics:
Expected Point Value (EPV) — From any game state (score, server, returner, surface, pressure), the probability the server wins the point. Trained on 1.88M points via gradient-boosted trees (AUC 0.847, temporally validated on 2025-26 data).
Shot Value — The marginal EPV change attributable to each individual shot. Aces and winners cluster at +0.8 to +1.0; unforced errors at −0.6 to −0.9. Verified against the telescoping identity: shot values across a rally sum exactly to the point outcome.
Decision Value — A principled decomposition of shot quality into decision and execution. A serve aimed wide that clips the net is bad execution but potentially optimal decision. This separation is what makes the metric actionable for coaching.
Aim Gap ⭐ — The headline metric: points per 100 service points a player forfeits by aiming suboptimally. Estimated causally via Augmented Inverse Propensity Weighting (AIPW) with doubly-robust estimation.
Tour-level finding: Players forfeit +0.73 pts/100 serves (95% CI [0.41, 1.05]). Right-handed vs right-handed: +0.87 (CI [0.54, 1.20]). Under high pressure: players shift −6 percentage points toward safer serves, costing −1.50 pts/100 (z = −3.75, p < 0.001).
The application delivers this through five interactive screens:
- EPV Rally Trace — A 12-shot Federer-vs-Djokovic rally with live EPV trajectory
- Leaderboard — Hierarchically shrunk per-player Aim Gap rankings with confidence intervals
- Aim Map — Per-player serve-aim distributions decomposed by court side
- Simulator — Real-time counterfactual serve-mixture adjustment
- Counterfactual Rally — "What-if" shot-level scenario analysis with bootstrap confidence intervals
How we built it
A two-server production architecture — because the judging criteria demand a live product, not a static notebook.
Data Layer
- 1,877,757 points parsed from the Match Charting Project (CC BY-NC-SA 4.0) via a streaming parser (
src/parsing/) that handles missingness codes, multi-line point entries, and era-spanning format inconsistencies - Structured into a 22-feature schema, stored as compressed Parquet for analytical workloads and JSON for frontend serving
EPV Engine (Phase 3)
- XGBoost classifier with 20 features: game score, set score, player identities (one-hot), surface, court side, pressure indicators, break/set/match point flags
- Temporal holdout on 2025-26 data (no leakage); match-grouped cross-validation
- Isotonic calibration on the validation fold — calibration error ≤3 percentage points across 10 probability bins
- Gated hard: AUC ≥ 0.83, log-loss ≤ 0.50. Nothing downstream runs until this passes.
Causal Inference (Phase 5)
- Augmented Inverse Propensity Weighting — the gold standard for observational causal estimation. Propensity scores from multinomial logistic regression (serve aim ~ 12 covariates); outcome model from gradient-boosted trees. Doubly robust: consistent if either model is correctly specified.
- Placebo test: permute treatment assignment, re-estimate. If the placebo Aim Gap is indistinguishable from zero, the identification strategy is valid. Ours: z = 3.5 → rejected.
- E-value: 2.1. An unmeasured confounder would need a risk ratio ≥ 2.1 with both treatment and outcome to fully explain away the effect.
- Bootstrap-by-match confidence intervals (500 replicates) to preserve match-level clustering.
Counterfactual Engine (Phase 9)
- Serve-aim counterfactuals: bootstrapped from 172,980 first-serve states with stratified resampling
- Shot-outcome counterfactuals: bootstrapped from 858,735 shots, filtered by striker identity and EPV proximity
- Rally propagation: EPV deltas decay forward at 0.3 per shot, capturing the diminishing influence of earlier decisions
- All estimates carry 95% bootstrap confidence intervals — a first for tennis counterfactual analysis
Backend
- FastAPI on port 8001 — 9 REST endpoints, zero model inference at serving time (all precomputed JSON + closed-form linear interpolation for the simulator)
- Counterfactual engine computes on-demand with cached bootstrap estimates
- CORS-enabled for frontend proxy
Frontend
- React + TypeScript + Vite on port 5173 — 5 screens with Recharts visualizations
- API proxy to backend (zero CORS issues, single-origin in production)
- Interactive: click any shot in the rally trace → counterfactual outcomes load in real-time
Validation Stack
- Cross-sport benchmark against NBA Two-for-One, soccer xG, and rugby penalty EP
- DML convergence analysis against Ma & Zou (2025): 60% agreement, 40% unique contributions
- 5 pre-registered hypotheses with Bonferroni correction
Challenges we ran into
1. Sandboxing at scale. The states data hits ~173K rows after filtering; the full shot-level data is 858K rows. Initial attempts to hold both in memory alongside bootstrap replicates triggered OOM kills at ~2GB. Solution: chunked bootstrap with stratified reservoir sampling — never materialize the full resampled dataset.
2. The positivity violation. Some players never serve to certain locations in certain game states. Naively estimating counterfactual EPV for a body serve that a player has never thrown in a break-point ad-court situation is extrapolation, not inference. Solution: abstention. If the propensity score for a treatment arm drops below 5%, we return "insufficient data" rather than fabricate a number.
3. Causal identification without experimentation. We're estimating the causal effect of serve aim on point outcomes from observational data. Players choose where to aim. The entire identification strategy rests on the assumption that, conditional on 12 observed covariates (score, surface, court, pressure, player identities), serve aim is as-good-as randomly assigned. This is a strong assumption. We defend it with: (a) the placebo test, (b) the E-value, (c) overlap diagnostics (SMD < 0.1 for 11/12 covariates), and (d) restricting counterfactuals to local mixture perturbations (±10pp) where the assumption is most defensible.
4. The body-serve SMD. One covariate — body serve propensity — had a standardized mean difference of 0.563 between treatment groups, failing the balance check. Rather than silently proceed, we flagged this, reported it transparently, and restricted the body-serve causal claims to exploratory status. Intellectual honesty over clean numbers.
5. Frontend-backend integration under deadline. Two servers, a proxy layer, TypeScript type strictness, and a dev-server host-allowlist that rejected the sandbox origin. Debugging required coordinating Vite config, FastAPI CORS, and the API client simultaneously. Solved by enforcing allowedHosts: true and a clean proxy configuration.
6. Multiple testing. Five pre-registered hypotheses means inflated family-wise error. Bonferroni correction at α = 0.01 killed H3 (experience) and made H5 (handedness) fragile. We report all five — including the null and fragile results — because pre-registration without transparency is just post-hoc storytelling with extra steps.
Accomplishments that we're proud of
Causal rigor in a domain that doesn't deserve it. Most tennis analytics projects stop at descriptive statistics — "players who serve wide win more." We went further: we estimated causal effects with doubly-robust inference, validated them against a placebo test, bounded unmeasured confounding with an E-value, and reported confidence intervals that exclude zero. This is the methodological standard of a peer-reviewed paper, applied to a Devpost submission.
The pressure finding is genuinely novel. No prior tennis analytics work has quantified the serve-aim shift under pressure with causal methods. The −6 percentage point shift toward conservative serves under high-leverage situations (break points, set points, match points), costing −1.50 pts/100, is a finding a coach can act on tomorrow morning.
Zero files in the root directory that don't belong there. Three files: README, Makefile, requirements.txt. Every other artifact lives in its proper subdirectory. Every diagram is SVG and PNG. Every document is cross-linked. The repository structure itself is a statement about engineering discipline.
First tennis counterfactual simulator with bootstrap CIs. SportsNGEN (2024) generates rallies with transformers but provides no causal guarantees. We generate counterfactuals with principled uncertainty quantification — every estimate comes with a 95% confidence interval, every delta is statistically grounded.
27 high-fidelity visual assets. Not clip-art. Not stock photos. Custom SVG diagrams — architecture, pipeline, court geometry, causal framework, cross-sport comparison, pressure analysis — all rendered at 2× resolution for retina displays, with a consistent design system (dark theme, tennis green primary, full color tokens documented).
An honest limitations section. Eight limitations, published prominently. No serve speed data. Selection bias in charted matches. Returner adaptation held fixed. Observational, not experimental. We didn't bury these — we built them into the methodology (abstention for positivity violations, local perturbation bounds, E-value for unmeasured confounding).
What we learned
Calibration is the deliverable, not AUC. An EPV model with 0.87 AUC but miscalibrated probabilities is worse than a 0.83 AUC model with perfect calibration — because every downstream metric (Shot Value, Decision Value, Aim Gap) inherits the calibration error. We spent more time on isotonic regression than on hyperparameter tuning, and it paid off in every subsequent phase.
Pre-registration is a superpower. Committing to five hypotheses before seeing results forced intellectual discipline. When H4 (ad-court gap > deuce-court gap) came back null, we didn't p-hack our way to significance — we published the null result, which strengthened the credibility of the findings that did survive correction.
Causal inference in sports is harder than it looks. The tennis serving context is one of the cleanest quasi-experimental settings in sports analytics — discrete decisions, observable states, no simultaneous play. And still the identification assumptions require careful defense. This experience deepened our respect for the experimental ideal and our skepticism of casual causal claims in observational sports data.
Two servers are worth the complexity. A single HTML file would have been faster. But the two-server architecture (FastAPI + React/Vite) gave us clean separation of concerns, enabled the counterfactual engine to compute on-demand, and made the application genuinely interactive rather than a static dashboard. The debugging pain was real; the architectural payoff was realer.
Cross-sport benchmarking reveals your blind spots. Comparing against NBA Two-for-One and soccer xG forced us to articulate what makes tennis analytics different — and what it has in common. Tennis leads in validation rigor; basketball leads in sample size; soccer leads in adoption. Knowing where you stand relative to adjacent domains sharpens the contribution statement.
What's next for Aim Gap - The Open Expected-Value Model Tennis Never Had
WTA Extension. The women's charting corpus exists but is structurally smaller (~2.2× less coverage than ATP). The EPV engine generalizes cleanly; the causal model needs re-estimation with surface-specific propensity scores.
Temporal Counterfactuals. How does the optimal serve mixture evolve across a match? Early-game information gain vs. late-game exploitation is an open problem. We have the infrastructure; the next phase is a reinforcement-learning formulation on top of the EPV engine.
Opponent-Aware Policy Optimization. Current counterfactuals hold the returner fixed. A full game-theoretic treatment would model the returner's adaptive response — a minimax formulation where the server's optimal mixture depends on the returner's observed tendencies. This is the bridge from analytics to prescriptive strategy.
Real-Time Coaching Integration. The current application is post-match. A live version — ingesting point-by-point charting data during a match and updating EPV estimates in real-time — would transform the tool from retrospective analysis to in-match tactical support.
Serve Speed & Landing Location. The current model conditions only on aim direction (wide/body/T). Incorporating serve speed and precise landing coordinates (available from Hawk-Eye at Grand Slams) would decompose the Aim Gap into decision and execution components at the shot level — the full Decision Value pipeline, end to end.
Open-Source Community Model. The EPV engine is designed to be retrainable on any charted dataset. We're publishing the training pipeline, the schema specification, and the validation gates — so that any team with charted data (collegiate, junior, recreational) can reproduce the full pipeline on their own corpus.
--
Log in or sign up for Devpost to join the conversation.