AugmentedView

• 

  Inspiration

• 

  Connect to users

• 

  Users stream

Inspiration

AugmentedView was inspired by the way media press interview people through Skype and from different studios these days.

Although the live interviews in media is fine as it is now, but we believe it can be more interactive using Augmented Reality and Computer Vision algorithms.

DEMO

YouTube

NOTE: the audio stream in the YouTube video went off due to a bug in iOS 11's screen recording feature.

What it does

AugmentedView is an iOS app that simply allows you to find and connect to live streams on an RTMP server and display the video stream in a real-world space using Augmented Reality.

How we built it

First, we setup the backend using node.js and websockets.

Second, we setup an RTMP stream using nginx RTMP module to push and pull stream from the client side.

Third, we built the iOS client side using Sockets IO and LFLiveKit to capture the audio and video stream from the iOS device.

Fourth, we used FFMPEG to play the RTMP Live stream in real-time.

Finally, we used ARKit with SpriteKit to display the video stream in a real-world space.

Challenges we ran into

Our biggest challenge was setting up the RTMP server due to some issues with network at the hackathon venue. However, we figured out a way to use a Wifi Hotspot to stream the RTMP Server locally and record the demo!

Accomplishments that we're proud of

1. Setting up an RTMP server in less than 24 hours.
2. Creating an FFMPEG player that plays a video stream in real-time.

What we learned

We have learned a lot while working on this project and we developed our skills in computer networks.

What's next for AugmentedView - Media Interviews using AR

We will keep developing the technology we've built to give the users a better experience.

Built With

• arkit
• hls
• javascript
• nginx
• node.js
• rtmp
• socket.io
• swift
• websockets