Fans are the most important part of a sports game. Getting fans to buy a ticket is half the problem but keeping them engaged is often overlooked. Why not give them a way to interact with the game through Augmented Reality? That's where Augmented Sports comes in.
What it does
The idea is that by collecting game data through a variety of methods, we can provide more insight into players and a game's tactics.
Using Computer Vision, we are able to overlay what is seen through the user's camera with player name, position, and important stats in the current year or career. This information is displayed above the player. Presenting the player stats helps people who don't know as much as the team changes their involvement throughout the game and helps visually impaired goers to see the game in their own way.
How we built it
We built Augmented Sports using ViroReact, a platform for developers to rapidly build AR apps using React Native.
The computer vision components were created with Python but we were unable to finish them.
This video demonstrates a great deal of what we had in mind in regards to the methods of data collection: https://drive.google.com/file/d/0B9WyrfZ8p4DuZUc4WGx5YnZSdWc/view?pref=2&pli=1
That is from this repo: https://github.com/binhnguyennus/football-tracking
We would utilize this technique before applying our AR approach.
Challenges we ran into
We were unable to use Expo, which would be our normal choice for rapid prototyping with react native. It took us some time to discover ViroReact which is similar and oriented towards AR/VR applications which in the end seems to be better for our purposes.
Many of the computer vision concepts that we want to implement are simply infeasible to implement within the time constraints of the hackathon. Thus, we have narrowed our scope to simply implement parts of our idea.
What we learned
Many of our members were new to React Native & have gained considerable experience with the framework.
Creating new computer vision programs within 12 hours is a difficult task, especially if nobody on the team has created the particular type of program before.
What's next for Augmented Sports
- Track players on the field in real time. Allow for commentators to draw explanations on the top down view of the field, which will convert to the AR view of the field.
- Fan Interaction / Experience
- Allow for chatting such that 3D AR chat bubbles float through the air. It can be user written or for less accountability, they can be predetermined (approved) phrases, emoticons, memes, gifs, etc.
- Chatrooms (filter people based on chatroom)
- Crowdsource Song Requests (if enough fans chip in to a money pool for it, then they can request songs - ones that support their team in particular)
- Modes, Filters, & Permissions (look at VRChat for reference)
- Virtual advertisements.
- Have list of players on your phone UI. If you press on a player, then an AR marker will appear above their head to show where they are on the field.