SoulPets

Narrative-based VR application to help users cope with pet loss

Comment

Inspiration

“In intimate relationships, we reshape ourselves over time to accommodate an interest here, a passion there, stretching, twisting, and changing until we fit together like the pieces of a puzzle. Then, at a loss, we discover our own incompleteness, and see how clearly our sense of self relied upon another’s presence.” – Ever so Humble, Linda Weltner

As of 2023, 66% of U.S. households (86.9 million people) own a pet, with a large number of pets having been adopted during the pandemic. Being pet owners ourselves, we were surprised by the lack of resources available for helping individuals dealing with the loss of a beloved pet.

We all consider our pets to be cherished family members, yet when they pass, we are expected to quickly move on, because there is no way the loss of a pet can compare to the loss of a human family member or friend.

The anecdotes we read online and responses we received from interviewing friends and family members told another story, however, one in which many felt unable to express their grief or felt lost in alone in managing their feelings of grief.

The loss of a pet is something that affects individuals at all stages of life, ranging from a young child losing a family pet that has always been around, or an elder losing a longtime companion.

What it does

SoulPets is an engaging interactive experience in which a user is able to reconnect with the ‘soul’ of their deceased pet. Drawing from the five stages of grief, the user is able to enter a space in which they are gently guided through the healing process alongside their pet.

The story begins with the player arriving on Soul Island upon which they are greeted by the soul of their pet. From there they arrive at the main campsite where they are able to decorate the campground, garden, and more. From the campsite, the user has the option to explore the 5 areas that each represent a stage of grief. In these areas, users will express their feelings in various ways, such as talking to an AI-backed chatting soul in the Talking Room, throwing and breaking objects in the Anger Pit, and rewatching the video clips of their beloved pets in the Memories Theatre.

After the 5 stages of grief, the user will come to an acceptance stage. Here the user will send their Soul Pet off through a portal which marks the beginning of the Soul Pet's adventure. The user may place various objects such as food and toys on the shrine which their Soul Pet will bring with them on their journey. Once in a while, the Soul Pet will send postcards back to the user with photos and messages. All of the postcards, along with tips and resources to aid with the grieving process will be posted on the bulletin board in the main campsite. Users can always come to the bulletin board to reminisce.

How we built it

To create the dreamy space of Soul Island and various locations within SoulPets, we produced 3D assets in Blender and AI-generated skyboxes and assembled them within Unity. The player interactions with these objects, the physics, and the corresponding logic were defined using C#. Local databases such as SQLite are used to keep track of various statuses.

In SoulPets, users may upload their pet photos, videos, and audio through a simple 2D Hub. To sync across the 2D Hub and Unity, we use cloud storage services such as AWS as an intermediate database. When users upload their files in the 2D Hub, the files will be stored in the cloud database, which will then be connected and downloaded from the Unity side. The photos will be used when sending postcards; the videos will be used in the memory theatre, and the audio will be played when users interact with their pets. This feature allows users to feel their pet’s presence in a more realistic way.

The talking room is a space where users can express their feelings and talk with someone. The user’s speech will be fed into a voice chat API or a chain of Voice recognition, OpenAI API, and Text-to-Speech services. Through these 3rd party APIs, we will get AI-generated responses in audio and deliver them back to the user.

Challenges we ran into

The representation of the pet soul is a concern we have during the early stages of brainstorming the application. At first, we want the pet soul to look like the user’s pet. However, that will require a lot of preliminary steps before the actual healing experience. Asking too many questions (What pet animal do you have? What color is it? etc.) will very likely pull the users out of the immersion. After rounds of discussion, we decided to make this step interactable too. First, we start with a blob-shaped soul. After the user has spent a considerable amount in the application, we analyze the type of pet they have based on the materials they uploaded, the objects they put in the shrine (such as pet food and favorite toy), and adjust the look of the soul (such as growing a pair of cat ears and a cat tail).

Accomplishments that we're proud of

Most of us didn't have any XR experience before participating in the Pico Dev Jam Hackathon, and we're proud that we are able to finish this project and make a simple demo to showcase the prototype.

What we learned

  • How Pico controller and hand-tracking interactions are supported
  • Connecting third-party AI APIs with Unity
  • How to make storytelling more effective in a VR setting

What's next for SoulPets

Currently, SoulPets only supports controller actions. However, we’re also planning to switch all interactions to hand tracking and body tracking since they will allow for a more immersive experience and will open up more possibilities with interactions such as waving, petting, and hugging.

To enhance the communication in the talking room, we will fine-tune the AI model by training it with other information collected in the application earlier, such as user-uploaded materials and their previous time spent in the stages of grief. In the future, we may also include a voice journal system using a local SQLite database where users can record their feelings and come back to them later.

Built With

Share this project:

Updates