The media landscape is changing fast. In order to keep viewers engaged, studios and other media outlets need to create more engaging content.
Augmented Reality is on of the most immersive and engaging mechanisms for content, but the missing element to bring that out into the physical world at scale has been the lack of proper network connectivity.
What could content -- perhaps an episode of the popular show Game of Thrones -- look like in the future when 5G and AR are working together at scale?
What it does
My project is an 'episode' of a show (storytelling) carried out in the physical world through AR. The user can walk along the streets of Los Angeles while engaging with the content and see the story unfold.
The low latency of 5G allows for real-time location-based content, and the high bandwith provides the opportunity to render 3D models dynamically from a server, as well as stream high-definition video into the AR scene.
The end result is an incredibly engaging, immersive entertainment experience.
How I built it
I used ARKit and libraries built on top of it.
Challenges I ran into
Filming on my iPhone on the streets of LA was tricky - pedestrians thought I was filming them when in reality I was filming the AR content as seen through the phone.
Accomplishments that I'm proud of
The product works as an experience - there are issues with performance on current iOS devices, as well as framerate limitations but the end result is the seed of something that I believe will help define entertainment consumption (and thus entertainment creation from media entities) into the future.
What's next for GoT 5G?
Working with even more immersive set pieces.
I also want to use computer vision models to recognize the scene in front of the user, and dynamically place content based on what the user is looking at. The low latency of 5G networks allows the user's device to preprocess and get inference predictions from a cloud service, usually resulting in much larger / accurate predictions (vs. on-device inference).