EDGE-VAR 2021 (IoT-assisted Referee) solution for Soccer based on AWS Wavelength and Verizon. Video-assisted Referee (VAR) has been used systematically in the Bundesliga since 2017 and at a FIFA-level since 2018. Current solution still based on the TV-transmission experience, humans review HD-footage taken by the broadcaster cameras. Controversial decisions still occur and games are paused for several minutes awaiting VAR-decisions. A new solution, EDGE-VAR 2021 (IoT-assisted Referee) will not be dependent on the broadcast footage but instead on a large number of smaller cameras placed on the pitch, on the players (who already wear GPS for athletic tracking) or even on the football itself. Controversial situations can be decided by triangulation of this large amount of image and telemetry data. Edge computing and 5G will be used to provide analysis in real time, cutting short pauses or maybe even eliminating them by feeding decisions to the referee. Considering the premium paid on broadcasting and advertising, there is a strong business case for reducing pauses. This project focusses on what is technologically possible, adoption by referees, players and fans is a separate issue. The application can be introduced on an experimental basis as decision support rather than as the ultimate authority. The following use cases are today decided by visual inspection of TV footage by VAR-team. Here is how they will be analyzed by EDGE-VAR 2021:
- was the player off-side? In the new solution, triangulation of telemetry data from the player's own GPS device, feed from small cameras on the sidelines or above the field.
- was the ball in? Analyze telemetry from the ball and from small cameras on the goal posts.
- Controversial no foul. The current rules mean that the referee will allow play to continue, wait for a natural play stop and then analyze the situation. IAR will provide a real-time decision or decision support while the ball is still in play. Technology: triangulation of several data sources both visual and telemetry is used today e.g. in self-driving cars. A classic cloud solution will be too slow, there is too much data to be communicated, processed and send back to the field. Instead, a solution based on 5G and edge computing will have faster processing times and still draw on resources from the cloud, for instance downloading statistical models. Bandwidth requirements will not be excessive since unlike VAR, IAR will not rely on UHD-footage but on flatter streams of pictures and data. 5G is already part of the fan experience in most stadiums. The pilot project could be developed together with Major League Soccer in the US and installed in one of the 10 cities in the US where AWS Wavelenght zones based on Verizon 5G are available. When European coverage is available, the AWS technology partnership with the Bundesliga should be brought into play.
What it does
Current prototype predicts goal/no goal based on images
How we built it
Jupyter Notebook, sci-kit learn, RandomForestClassifier using 229 images of goal/no goal-situations
Challenges we ran into
Benefits of using AWS Wavelenght on Verizon 5G will be more evident when analyzing large datasets comprising not only images but also positional and physical data (e.g. a goal post being impacted by the ball).
Accomplishments that we're proud of
VAR was introduced to make the game more legitimate but remains controversial.
What we learned
Soccer is complex, rules and people interact in complex way. Technology must be introduced in the right way.
What's next for EDGE-VAR 2021
Apply for funding to perform real-life testing with the involvement of an actual referee. Make architectural decision on whether the app should be based on machine learning of past data or on a formal rule engine. The latter option would make it more akin to a gaming app taking real-life footage as input and building explicit 3D-models of player and ball movement.