Tele-Room

Inspiration

Tele-Room is an app for iOS and Android that allows for AR interactions in a shared livestream video. We hope that this can help people connect better across different spaces, and especially to help people who are less mobile connect to friends and family who are away.

What it does

We imagined possible two ways this app could be used:

A person in a hospital bed streams a video of their surroundings from a tablet or phone. Friends can join into the stream and place virtual 3D objects around the room to decorate it. The person streaming can tap on these objects from their phone to interact with them, with audio and haptic feedback. These decorations can be saved and viewed by the person after the stream. In this way, they can have meaningful, persistent interactions with the ones they love, despite being away and unable to move from their room.

An ill person who cannot leave their room can watch a livestream of their friend taking a walk. This person can drop different markers and virtual objects on the path as their friend walks, taking part in the activity and having the ability to interact with an environment that is otherwise inaccessible to them.

How I built it

Viewing and Streaming Experience - both users can place and manipulate virtual objects, streamer controls the video stream and keeps track of the virtual space.

We are developing this app on Unity, using ARKit for iOS and ARCore for Android. We have integrated Cloudinary as a way to store 3D models that can be placed in the space. We hope to also use Cloudinary as a way to store 3D data from livestream sessions, so that users can save and review scenes they have decorated in the past.

Challenges I ran into

Unity is very unstable :(

Accomplishments that I'm proud of

But we managed to get it work (to some extent)

What I learned

To not use Unity for hackathons

What's next for Tele-Room

Future additions to the app:

• More visuals
• Better depth info
• Design for users with disabilities
• Letting viewers move and act in virtual space with motion-tracking
• 360 video

Built With

• unity