Review Room

Like an NFL red flag, cricket's replay challenge is a bet. Review Room prices it in win chance from 5,132 real challenges: unless you're ~92% sure the umpire is right, challenge.

Comment

Inspiration

Major League Cricket, 2026. San Francisco need 12 off 7 balls. The umpire says not out. Los Angeles has 15 seconds to decide whether to throw cricket's red flag.

Cricket's challenge system (DRS) works like the NFL red flag or the MLB manager's challenge. Teams get only two failed challenges per innings, so they hesitate. But nobody had priced the bet. For that call: 29.2 points of win chance at stake, a failed challenge would cost 0.01. Los Angeles needed just a 1-in-2,760 belief the umpire was wrong. They challenged, it was out, they won.

We wanted to know how often a challenge is that cheap.

What it does

Review Room prices both sides of every challenge in win chance, the "% to win" line from NFL and NBA broadcasts. We did it for 5,132 real challenges across 11 pro T20 leagues (2017–2026).

Typical
A disputed call is worth 5.5 pts
A failed challenge costs 0.08 pts
Confidence needed 1.3% (1-in-77)

Teams win 27% of challenges, far above that bar, so they're holding back. 463 batters walked off "out" with a challenge still unused.

  • Cheat sheet: one rule for the locker room: unless you're ~92% sure the umpire is right, challenge.
  • Review Room: replay any real challenge, set your confidence with a slider, reveal the verdict.
  • New to cricket? A four-point primer built on NFL and MLB comparisons.

How we built it

  • Data: 8,897 T20 matches (2.04M balls) from Cricsheet.
  • Win-chance model: gradient boosting (scikit-learn) with every feature measured against the league's recent par score. Trained on pre-2024 games, tested on 2,813 unseen matches. It beats the textbook logistic model and is well calibrated (says 70%, wins ~70%).
  • Cost of a lost challenge: we rebuilt "challenges left" for every match, then measured what a held challenge is worth later.
  • Decision rule: with \( G \) = what the call is worth, \( L \) = what a lost challenge costs, challenge when

$$ p > \frac{L}{G+L} $$

  • App: static site, plain JavaScript, hand-built SVG. No server, no login. 11 tests check the browser matches the Python.

Challenges we ran into

  • The IPL broke our first model. Scores of 200+ were outside the training data, and tree models can't extrapolate. Measuring runs against par fixed it, like judging NBA points against that season's pace.
  • The data never says how many challenges are left. We rebuilt it and checked it against the IPL's real 2022 rule change.
  • We couldn't predict which challenges succeed. Error 0.5602 vs 0.5603: no gain. Only the player sees the ball, so the app asks for their confidence instead of guessing.

Accomplishments that we're proud of

  • Turned a gut call into one rule a coach can use.
  • Validated on 2,813 future matches with confidence intervals, not a demo.
  • Published our negative result, and it made the product better.
  • Holds under stress: even at 3× our cost estimate, 81% of decisions needed under a 1-in-10 belief.

What we learned

  • Pricing a decision beats predicting an outcome. We didn't need to predict the replay; the bet is that lopsided.
  • In sports data, context is everything. Raw numbers broke; par-relative ones held.
  • Explaining is half the product. If a judge who's never seen cricket can't follow it in 30 seconds, the math doesn't matter.

What's next for Review Room

  • Ball-tracking data to estimate confidence automatically.
  • Live broadcast mode: a "should they challenge?" graphic during the 15-second clock.
  • Team audits for Major League Cricket franchises.
  • Other sports. The same math prices an NFL red flag or an MLB manager's challenge: what the call is worth vs. what the challenge is worth.
Share this project:

Updates

Submission history