I thought that we analyze momentary events (items like diving or gymnastics) with a score relatively pattern predicatively and machine learning can pick up on that and predict if the score was well enough in many ways.
What it does
It takes a video of a performance of a dive and tells you whether they scored above a threshold or not with high precision in real time.
How I built it
- Get once every few frames of the video
- Color transform and smooth the frame so that you detect the humanoid body
- Get a speed up robust features transform (SURF) with circular distances at the keypoints
- Form a feature vector sorted by the largest distance to smallest distance, encoding the pixels as a linear array. Apply principal components (namely take the N biggest values) and put them into a feature vector.
- Use a classifier trained on images and relative scores to predict whether the score was over or under
- Store that frame's prediction in memory and then to make a valid prediction look at the last few frames
- Report and sit back
Challenges I ran into
- Speed of encoding
- Right color boundaries
- Which transform
- Encoding feature vector
- Which classifier on which parameters
- Memory frame
- Machine Learning not converging or not generalizing or a whole host of issues
Accomplishments that I'm proud of
- It works with about a 80% accuracy for the test video. More data and testing are needed
What I learned
- Machine learning is really really cool, sometimes the most simple methods are all you need to properly identify patterns for good and bad judging.
What's next for The Judge
Possible Applications: Use this algorithm build on the idea of predicatively gauging score through simple videos so those practicing alone can have a reliable benchmark either on diving or gymnastics or any sport with a level of skill and judging needed or even a 1-0 encoding of good or bad. (Equestrian sports included)
The algorithm also lend's itself to find ways to adjust athlete's form -- giving them a state of the art way to fine tune their performance and bodies.
Apart from that, a judge would simply need to expand the size of the training set and put in a multi classification system with varied parameters to get a reliable score estimate.
The algorithm runs in real time on a simple laptop so scaling to a map-reduce or simply witnessing it run in real time is enough
There are millions of sport and other clinically fatigue based injuries and deaths that we can prevent with a ubiquitous agreement on "good form" in the interest of athlete health. Subjective views of safety can be combined with analytic to ensure some measure of safety or raising red flags when need be (such as boxing matches or long distance swimming) real time lightweight analytic provide that.
As the cliche goes, the possibilities are simply endless. The code itself is of public domain but the algorithm (and further developments) are reserved exclusively to me.