442ools

442OOLS uses AI vision to decode split-second sports decisions from video - tracking player postures, body feints, and predicting plays right before they happen.

Comment

Inspiration

Football has always been the most data-rich sport that paradoxically gives its practitioners the least data in real time. Coaches freeze-frame film sessions to debate the one pass that wasn't played; analysts spend hours in Wyscout tagging events that happened three days ago. We kept asking the same question: what if a manager could see that analysis — live, frame by frame, while the match is still happening?

The spark came from watching Kevin De Bruyne orchestrate a City build-up and wondering: could a system prove, mathematically, that the through-ball into the channel was the right call — and show it on screen the moment the decision needed to be made? From Mbappé's body feints to a Nico Williams vs John Stones duel, every elite duel is a physics problem dressed in boots. We wanted a tool that could decode it.

442ools is named after the classic 4-4-2 formation — a tribute to the tactical geometry that underlies the beautiful game — and "ools" for the tools we always wished existed.

What it does

442ools is a real-time football intelligence platform with two parallel analysis engines:

1 · Tactical Decision Engine (Pass / Shot Intelligence) For every frame in which a player has possession, the system evaluates and ranks all possible actions, then surfaces only the single best play:

Pass typology: ground precision pass, through-ball, lob/cross into the danger zone Expected Threat ($xT$): each candidate destination is scored against a calibrated $16 \times 12$ pitch-value grid. The $xT$ gain of a proposed action is: $$\Delta xT = xT_{\text{destination}} - xT_{\text{current}}$$

Through-ball success probability: a physics model estimates whether a played ball will reach its target before a defender can intercept, accounting for defender reaction speed, acceleration cone, and GK sweeping agility: $$P_{\text{success}} = \sigma!\left(\frac{d_{\text{clearance}}}{v_{\text{ball}}} - \frac{d_{\text{defender}}}{v_{\text{defender}} \cdot r_{\text{reaction}}}\right)$$

Space zone highlighting: convex-hull polygons shade exploitable half-space pockets directly on the video frame in real time Lob / cross arcs: quadratic Bézier curves show ball trajectory with a peak apex that reflects crossing distance and angle A shot threat overlay fires simultaneously if the shot xT exceeds a threshold — the only case where two vectors appear at once 2 · Dribble & Duel Biomechanics Engine For close-up 1v1 footage, the system runs YOLOv8-Pose on every player and:

Renders role-coded glowing skeletons: 🩷 Pink = ball carrier, 🔵 Blue = defender, 🟠 Orange = goalkeeper Computes 17 COCO keypoints per player and derives joint angles — hip lean, knee flexion, ankle dorsiflexion, stance width — to classify posture state Produces a real-time joystick/directional burst arrow showing the most likely take-on direction based on the attacker's centre-of-mass trajectory and foot plant angle Detects body feints: a sudden lateral hip rotation without corresponding foot displacement is flagged as a directional deception move Scores the defender's reaction — whether their weight transfer commits them out of position before the dribbler's actual direction is declared 3 · Live Web Studio A FastAPI backend streams all events to a browser dashboard featuring:

Real-time 2D pitch radar with player positions, pass vector, and space zones Live event feed: every pass suggestion, shot opportunity, and dribble event timestamped to the frame Analysis card per player: xT, suggestion type, success probability badge

How I built it

Input video │ ▼ ┌─────────────────────────────────────┐ │ YOLOv8 (custom fine-tuned) │ Player · GK · Referee · Ball detection │ + ByteTrack multi-object tracking │ Jersey-colour team stabilisation └──────────────┬──────────────────────┘ │ ┌─────────┴──────────┐ ▼ ▼ ┌────────┐ ┌───────────┐ │ Dribble Pose │ │ PlayPredictor │ │ Engine │ │ (Tactical Engine) │ │ YOLOv8-Pose │ │ xT Grid + Physics │ │ 17 keypoints │ │ Pass Typology │ │ Joint angles │ │ GK Agility Model│ └───┬────┘ └─────┬──────┘ │ │ └──────────┬───────────┘ ▼ ┌─────────────────────┐ │ PitchTransformer │ Homography calibration │ (pixel ↔ pitch) │ Forward + inverse H └──────────┬──────────┘ ▼ ┌──────────────────────┐ │ detect_track.py │ OpenCV overlay renderer │ Bezier curves │ Probability badges │ Convex hull shading │ Skeleton glow lines └──────────┬───────────┘ ▼ ┌──────────────────────┐ │ FastAPI + JSONL │ Real-time event stream │ Web Dashboard │ Pitch radar · Cards └──────────────────────┘

Model training: We fine-tuned a YOLOv8 detector on a custom-labelled dataset of broadcast football frames across multiple leagues, leagues, and camera angles to reliably separate players, goalkeepers, referees, and the ball — each class requiring distinct treatment downstream.

Coordinate system: A homography matrix $H$ maps pixel coordinates from the broadcast camera view to normalised pitch coordinates $(x_p, y_p) \in [0, 105] \times [0, 68]$ metres. The inverse $H^{-1}$ is used to project tactical suggestions back to pixel space for rendering.

$$\begin{pmatrix} x_p \ y_p \ 1 \end{pmatrix} = H \begin{pmatrix} u \ v \ 1 \end{pmatrix}$$

Pass physics: Defender intercept cones are modelled as circular reach zones that expand over time. The clearance gap is the perpendicular distance from the nearest defender's projected position to the pass trajectory at the estimated moment of ball arrival.

Challenges we ran into

  1. Homography instability across camera cuts Broadcast football constantly cuts between wide, tactical, and close-up cameras. Each cut invalidates the homography. We built a calibration system using pitch line intersections (penalty arcs, centre circle, corner flags) as reference keypoints, but getting this to auto-detect reliably across different broadcast feeds — with varying aspect ratios, lens distortion, and replay overlays — was the single hardest engineering problem.

  2. Skeleton identity swapping YOLOv8-Pose runs on each frame independently. In congested dribble situations with 3–4 players in frame, track IDs from ByteTrack would occasionally jump between bodies mid-sequence, causing the attacker's pink skeleton to "teleport" to a defender mid-dribble. We solved this by anchoring pose identity to the ByteTrack bounding box IoU rather than re-assigning per-frame.

  3. Defining "best play" without ground truth There's no ground truth label for "the correct pass to play." We had to construct a composite scoring function that felt tactically coherent — $\Delta xT$ alone would always recommend shooting from 60 metres. Balancing xT gain against pass success probability, space exploitation, and GK agility required extensive manual review against actual elite footage.

  4. Rendering speed Drawing glowing Bézier curves, per-keypoint glow halos, convex-hull polygons, and probability badges on 60fps 4K footage in Python/OpenCV is slow. We aggressively cached homography lookups, pre-computed xT grid values as a NumPy array for $O(1)$ lookup, and minimised per-frame Python loops to hit real-time throughput on consumer hardware.

  5. Screen recording incompatibility The macOS screen recording saved with a Unicode narrow non-breaking space (\u202f) in its filename, which caused every file I/O library — shutil, cv2, moviepy, shell cp — to silently fail with FileNotFoundError. Took embarrassingly long to isolate via repr() inspection.

Accomplishments that we're proud of

  • End-to-end pipeline from raw broadcast video to annotated tactical overlay, running on CPU without any GPU requirement
  • A physics-grounded pass success probability model that meaningfully differentiates between an easy 10-yard square ball and a diagonal through-ball into a congested backline — without any training data, purely from geometry
  • Role-coded pose skeletons that remain identity-stable across the full duration of a dribble sequence, even in contact situations
  • The space zone convex hull renderer, which visually communicates exploitable space in a way that any coach can read instantly — no data literacy required
  • A single-best-play policy that forces the model to commit to one recommendation per frame, mirroring the cognitive discipline we expect of the player — and making the output actually usable rather than an overwhelming overlay

What we learned

  • Computer vision and football intuition are both necessary — neither is sufficient. A technically correct model that recommends a 40-metre lob every time because the xT delta is marginally higher is useless. Domain knowledge must inform every hyperparameter.
  • Pose estimation is deceptively fragile in real-world sports footage. Occlusion, motion blur, and extreme joint angles (a goalkeeper full-stretch dive) regularly produce anatomically impossible keypoint configurations that break downstream angle calculations. Confidence thresholding and graceful degradation matter enormously.
  • Homography is the load-bearing wall of broadcast sports analytics. Everything — pass vectors, space polygons, GK positioning — is meaningless if the coordinate mapping is off by even a few metres. Getting this right before anything else is the right order of operations.
  • Rendering is a product problem, not just an engineering one. We iterated more on how to draw the suggestions (colour, thickness, label position, transparency) than on the underlying model, because a suggestion that's visually confusing is no better than no suggestion at all.

What's next for 442ools

  • Real-time live match input via RTMP stream (instead of pre-recorded video)
  • Automated homography calibration using pitch line detection (no manual keypoint selection)
  • Expand dribble engine to detect pressing triggers — moments when a defender's body shape and closing speed make a press viable

  • Team-level shape analysis: compactness, defensive block depth, width between lines

  • Multi-camera feed fusion — combining the tactical wide view with close-up player cams xT model personalisation: learn a player-specific xT modifier based on historical decision-making tendencies (e.g. De Bruyne's through-ball success rate is higher than the population baseline)

  • Export to coaching session format: clip all top-5 tactical moments from a match with annotated freeze-frames for video analysis

Long-term

  • Extend beyond football: the xT framework and pose biomechanics engine are sport-agnostic — basketball spacing, basketball on-ball defence, ice hockey forechecking are all natural extensions
  • Partner with an academy or semi-professional club for live deployment and feedback-driven iteration
  • Publish the custom broadcast football detection dataset to the open-source community

442ools is a love letter to the game — and a bet that the best tactical minds in football deserve better tools than a whiteboard and a marker.

Built With

Share this project:

Updates

Submission history