In May 2018 Google presented Cloud Anchors as a resource for doing shareable augmented reality. Many apps have been made from it already, creating shared experiences between devices. Albeit the ability to do this is rather new, we wanted to explore how it could enable not just shareable experiences, but be integrated into social experiences. Social experiences are described as the "next big thing" that AR will enable us to do. Social interactions happen between people, hence we see group settings as interesting to augment. So, working with a group setting, how can we introduce shareable augmented reality to both optimize and gamify the experience in the social context?
We present one of the possibly millions of ways to augment social experiences. In most restaurants, especially here in Tokyo, it can be hard to know what you're ordering. Menus consist mostly of small images, which doesn't necessarily represent the food very well. The social interactions in this space occur especially when the group orders - as menus are passed from person to person, and arguments are made for what type of food should be ordered. We want to augment this experience so we can help overcome the problem of representing and ordering food in a restaurant.
What it does
When opening the application, a surface is scanned and an anchor is placed on the surface. This creates a room that everybody with the app can access. Choosing from the menu, you can place food on the surface and share with everyone in the same augmented room.
How we built it
We're using a game engine called Unity to develop an AR application using ARCore and ARKit (without markers). Through Google's Cloud Anchors we were able to integrate sharing the AR between devices. By initializing an anchor, you create a room. Google Cloud enables us to store session information and allow users to join those sessions.
Challenges we ran into
Our initial pitch was to just look at sharing an AR experience. After some research, we discovered a few apps which already explores this. Thus we found a new focus in what is described as "AR's next big thing" - which is applications for augmenting social experiences.
Cloud Anchors is a quite new concept and technology. Its documentation is limited, therefore it caused some integration issues, which was resolved along the way to our MVP.
Accomplishments that we're proud of
- Integrating Cloud Anchors to create shared augmented reality.
- Making the restaurant menu card placeable
- Presenting the food from the menu, with details and ingredient lists
- Stable sharing of augmented visuals
- Working on particularities in the Japanese culture
What we learned
- Working with and integrating Cloud Anchors
- Learning new tools (unity, ARCore and more) that some team members didn't know before
- Teamwork and overcoming challenges together
- Integrating API's into an AR project
What's next for Zocial
Zocial is an app that could expand to more of the environment in a restaurant. For instance, bills and proceedings could be incorporated into the app. We could also make the app more generic, so it's possible to be used by more than one restaurant. This could be done by utilizing Artificial Intelligence in creating 3D objects of food orderings.
We believe that the possibilities for the technology behind Zocial are endless. Through creating an augmented social experience for making dining more social we are ready to take our concept to the next level. By using AR technology and cloud anchors, we want to take on new challenges in different industries, like education and gaming.
We see at least two ways shareable AR can be used for social experiences: Augmenting existing social settings or creating entirely new ones. With Zocial we applied the first way, while the latter could i.e. be used for social games.