For our study we are going to track the vitals of each player and compare that to how many baskets they make. By vitals we mean heart rate, and breathing which we can try to help the player control which then in turn helps them make more shots. By keeping the players' vitals consistent, there is less room for error which can stem from the player breathing differently or too fast of a heart rate leading to nervous behavior and sweating.
To test our experiment, we could gather a number of individuals and track their heart rate and breathing before they get on a court as a baseline. Once we record our data before our tests, then we would put these athletes through numerous activities to get their heart rate up and get them to start breathing heavier. Once we are done with this we would make each player shoot 120 shots (30 two pointers, 30 three pointers, and 30 free throws) and then gather their results. At the same time, we would send each player's vitals through the use of VZW 5G to our data collector on AWS so we can effectively and quickly highlight some numbers that may stand out in relation to how many shots that individual made or missed.
After seeing any numbers that stand out, we can go back to these individuals and point out any spikes that may have affected their shot and their shot process. If we can minimize any catalyst that affects one’s shot making, we can fix it and create a consistent routine and allow these players to shoot more effectively.
To track how many shots each player makes we will use a camera that is connected wirelessly to VZW 5G network that then sends that video stream to the AWS database and services to analyze the video stream such as facial recognition, shooting monitoring using machine learning, etc. Essentially each player will have a heart rate monitor tracking their heart rate, and also a motion sensor that tracks their movement. By tracking their movement, we can compare the data to their heart rate as our hypothesis is that the higher your heart rate, the slower your movements are because of factors such as fatigue and exhaustion.
Our three different trials, to ensure effectiveness, can take place during practice, during warmups before a game, and during the player’s actual game. Our ability to use 5G and AWS wavelength can help us send our data in real time to the coaches and the staff. The staff can use this to help plan out game strategy more efficiently and give players who might not be playing as well or are getting more tired a break so that coaches have the ability to win more games in the long run.
What it does
From the video streams of the footage of basket playing and smart watch which provides BPM(heart beat per minute), player performance data is collected to be used for coach or staff to use to pick up the best condition player before game. UE will send the video stream to EC2 on AWS wavelength through VZW 5G, the video stream is processed on the highly optimized EC2 instance for machine learning coming from VZW 5G with very low latency along high throughput. Multiple camera footages are able to be processed in real time.
How we built it
Capture videos of playing basketball and feed those into AWS EC2 in Wavelength over VZW 5G where videos are analyzed and KPI(Key Performance Index) are calculated. Two web client web apps can access video analytic server and KPI server. All are implemented in python. OpenCV, Yolov5, Flask, MediaPipe, PlotlyDash, Socket APIs are used to make server and client work for real-time wireless player performance analytics.
Challenges we ran into
Camera videos from the phone connected to NOVA is not possible, so we saved three videos with MP4 format, downloading those files into the phone connected NOVA, sending the MP4 files over UDP using python script running on Android Termux. Since it is read from the files and opening UDP sockets on the phone, it is very hard to sync three video files, so there were a lot of trials and errors to adjust latency and packet sizes on the python script.
Accomplishments that we're proud of
This is a real prototype end to end (phone <-> network <-> cloud ) implementation and test on real phone, wireless network, and cloud(Edge computing node) inspired, requested by a real basketball player. Basketball activities and collecting KPIs(2,3 point shots, heart rate, oxygen level, etc) are most complicated, but this prototype implementation gave us confidence that we can improve our prototype to apply to other sports and even commercialized, proving that we can easily develop any sport analytic server and client required of low latency, and high throughput on VZW 5G along with on the high power machine on AWS.
What we learned
AWS EC2 has very powerful image for a variety of machines(CPU,GPU,TPU, network optimization, etc) which we can implement any high throughput data and builtin machine learning on the high performance machine will make developers be free from the limitation of computational power. And the idea of edge computing implemented in wavelength over 5G gives developers more attractions to deploy realtime data analytics.
What's next for Real-time Wireless Player Performance Analytics
Enhancing machine learning algorithm and making the prototype framework realistic applied to golf practice. Just put phone camera to capture and analyze the golfer's practice by feeding the video images to video analytic and KPI servers on AWS wavelength via 5G, that we developed in this challenge running machine learning to detect pose and collect KPIs like swing, speed, clue pose, and compare those with those of pro golfer's position,and recommend to correct the positions in real-time which is possible as we implemented and ran our code to work on AWS wavelength over VZW 5G.