The metaverse is often framed as something that exists either adjacent or outside of the regular, physical reality we inhabit. In the future, perhaps, we will put on headsets and be transported to some magical place we could otherwise never reach. In contrast, our project focuses on the ways in which the metaverse can be used to enhance our relationship with the physical spaces we already inhabit and reveal things which may otherwise be invisible.
Much like Pokemon Go used AR to encourage people to go out and interact with their community, our project, Stone Stories, is focused around using augmented reality to mediate interactions between individuals in their communities. We wanted to give individuals the opportunity to build something in a place that was significant to them, and to let others share in that creation and add to it, an action that takes on added significance in the face of the continuing pandemic. By perpetuating an ancient practice (the building of cairns) through the use of cutting edge technology, we wanted to combine the past and the future to lower the barrier to community interaction
What it does
Participants who boot up the app see a cairn built collectively by other individuals who have been through the app. Users have the chance to take a stone and decide how to add it to the virtual cairn to leave their own impact. Users are able to customize their stones with images and audio, and others who visit this monument have the ability to inspect each stone to learn more about others who have come before.
The project disturbs the idea of a public monument by recounting the spirit of place of those who are not present and those who pass through and leave something behind. Since prehistory, stacked stones structures called Cairns have served many purposes – for navigation, as landmarks, places of gathering, spiritual symbols, burial monuments, and counting systems. They reflect long-lasting relationships to place and land, and political creativity.
How we built it
We used react-three-fiber, a wrapper for threejs, and WebXR interfaces to develop. We tested it through Quest 2 (for VR) and Microsoft's Hololens (for AR).
Challenges we ran into
Lots of build errors for our react-three-fiber based project. The code to grab and interact with the stones did not play well with the physics engine we tried to implement. At the end of the day, there just wasn't a lot of time!
Accomplishments that we're proud of
We made a working demo where people can drop stones, which are sent to a database and recreated when a new user enters the app. It's a strong indicator that the idea would work well in practice with some additional implementations like geolocation or plane detection.
What we learned
WebXR was a new technology for most of us, and we learned a lot about how deploying on the web comes with its own set of challenges that are very unlike building with a more traditional game engine like Unity or Unreal. We also learned about designing interactions around site-specific projects and some of the challenges that entails.
What's next for Stone Stories
We'll be adding more and more customization options for the stones that are deposited, like customized texturing and voice notes that other users can interact with, as well as support for multiple users interacting with the cairn at once. We also want to improve our ability to detect planes in AR.