Ai-Companion

GIF
(GIF) Pick Up
GIF
(GIF) Behaviours
GIF
(GIF) Commands
Adaptable Memory System

Inspiration

In an increasingly digital world, many of us find ourselves spending more time in front of screens, whether for work, study, or entertainment. Despite the convenience of technology, feelings of isolation and digital overload can take a toll. The need for personalized, emotionally engaging interactions is more important than ever.

AiCompanion was created to fill this gap. It serves as a digital companion that grows with its user, offering a friendly presence for moments of solitude and assisting with everyday tasks. By combining AI-driven interactions with a personality that adapts to become your ideal companion over time, AiCompanion aims to create a bond between user and machine that transcends simple utility, providing a dynamic, supportive experience.

What it does

AiCompanion is a desktop-based AI companion that offers a wide range of interactive features designed to enhance both productivity and personal well-being. The AI evolves based on user interactions, adapting its responses, mood, and capabilities over time. Some of the key features include:

Personalized Interaction: AiCompanion learns from the user’s behavior, preferences, and communication style, adapting its personality and tone to match. Whether it's offering reminders, sharing news, or providing a fun distraction, it tailors every interaction to the user’s needs.

Emotional Engagement: AiCompanion can detect emotional cues from the user’s language and provide empathetic responses, making it feel like a true companion. It can provide support, offer motivational phrases, or engage in lighthearted conversation based on how the user is feeling.

Task Assistance: AiCompanion helps manage daily tasks such as setting reminders, managing to-do lists, or providing useful information. Its adaptability means that it can even assist in novel ways, learning how to best serve the user over time.

How we built it

We built Ai inside of Unity in order to utilize Unity's Tools. These include the animation player to Handle's Ais animations or Unity Version Control which is faster than Git for Unity projects. All of the systems are built using C#.

For the AI, we utilized Whisper for Speech-To-Text, OpenAI for the Language Model, and ElevenLabs for Text-To-Speech. OpenAI models were implemented utilizing OpenAI-Unity. Ai's memory system utilizes prompt engineering to generate memory and sentiment logs in JSON form.

Figma was used to design the buttons and menus.

Challenges we ran into

Optimization. LLM models are expensive to run on a machine, hence to ensure compatibility we leveraged APIs.

ChatGPT API has no memory. Our approach was to create a JSON that stores a user profile and memories. We utilize two layers of API calls to ChatGPT: the first analyzes for potential updates to the user profile, the second layer then injects relevant information that Ai utilizes to give informed responses.

ChatGPT API can't provide realtime information. Our solution is depending on keywords, we injected information from a News API and Weather API to ensure Ai can give responses based on Realtime information.

ChatGPT can't touch your system. For assistant capabilities, messages are scanned for keywords. We utilized Unity system functions, Windows DLL, and OpenAI to provide assistant functionalities like adding to your calendar.

Accomplishments that we're proud of

The memory system. Figuring out how to format the user profile, how to scan for relevant information to update the profile with, and then effectively leveraging the user profile felt like the key which brought this project from chatbot to an intelligent companion.

What we learned

Learned how to break past OpenAI's limitations utilizing a locally stored memory system, prompt engineering, and clever design.

Learned how to leverage several APIs in tandem such as Open Weather with OpenAI to provide a more immersive experience.

What's next for Ai-Companion

Likely open sourcing and bringing this project to a greater community. We had specially designed Ai-Companion to be easy to expand thanks to modular architecture which should make modding support easy. There's also inbuilt language support through Whisper and Elevenlabs. If we get permission from the creator of VPet, we may be able to tap into their greater community giving a global AI-powered companion to everyone for free through platforms like steam.