PortaPet

Landing Page - Hero
Landing Page
Create PortaPet Page
Augmented Reality Pet
Augmented Reality Pet

Inspiration

Moving to a new country for post-secondary education made us realize how difficult it is to be away from loved ones, especially pets. We wanted to create a solution that helps maintain the emotional bond with pets even from a distance, offering comfort and companionship through an interactive virtual experience. PortaPet was created to make those separations easier by bringing pets to life through immersive technology.

What it does

PortaPet uses AI and 3D visualization to let users interact with virtual representations of their pets. It allows users to upload an image of a pet that gets converted into an augmented reality model and allows users to talk with their pet through AI-driven conversations and visualize cherished memories through immersive, lifelike 3D renderings. Whether users are away or dealing with the loss of a pet, PortaPet creates a comforting, interactive experience that keeps their bond alive.

How we built it

The front end of PortaPet is built using typescript in a Next.js framework, Tailwind CSS, Lucide React, and three.js, along with Spline and GSAP to create a 3D rendering and webxr app. In the backend, we utilized Vercel Blob to store images that the user uploads, which sends it to MeshyAPI to create a 3D rendering that the user submits that portrays the user's pet or loved one. We also used wake word functionality to start listening to prompts if they start with a keyword (the pet's name!) and Next.js's web-to-speech API to convert the user's speech into text, which is sent to various AI's, such as CloudFlare to determine the response to the user, as well as ElevenLabs to speak the response generated in a way that the loved one would.

Challenges we ran into

One of the key challenges was balancing the emotional aspect of the project with the technical limitations. Ensuring that AI conversations felt genuine and personalized required a lot of fine-tuning and trials. Additionally, integrating 3D renderings that were not only visually accurate but also interactive was difficult, especially when dealing with real-time user input and feedback. Ensuring cross-device compatibility and smooth performance also proved to be difficult.

Accomplishments that we're proud of

We are proud of creating a seamless and emotional experience that aids many users who may be suffering emotionally or psychologically. Developing an application that enables real-time interaction with virtual pets while maintaining high visual and emotional support was a major accomplishment. We also successfully combined multiple advanced technologies—AI, 3D visualization/XR, and real-time speech synthesis—into one easy-to-use platform.

What we learned

Through this project, we learned the importance of human-centred design, particularly when working on a project that blends technology with emotional well-being. We also learned how to implement and deploy extended reality apps, 3D rendering, and artificial intelligence to create smooth user experiences across devices and how to fine-tune the AI.

What's next for PortaPet

In the future, we plan to extend PortaPet's capabilities by allowing users to import multiple pets in one world and providing a memory system where users can re-experience events with their pet. We also plan on expanding interaction methods, such as integrating haptic feedback for a stronger connection to their pet!