Inspiration

Growing up, I loved the idea of virtual pets like Tamagotchi. You could feed them, play with them, and watch them grow. But no matter how much time you spent with them, they were still following a set of predefined rules.

That got me thinking. What if a virtual pet could actually learn from you?

I've always found reinforcement learning fascinating, especially the idea that an agent can figure out what to do through trial and error rather than being told exactly how to behave. But most RL demonstrations involve an agent navigating a maze, playing a game, or maximizing some numerical reward. You can see that the algorithm works, but it's difficult to connect with what is actually happening.

I wanted to make that learning process more personal and intuitive. Instead of watching an agent learn to solve a task, I wanted people to experience RL by raising their own artificial creature.

That's how Pip started.

What it does

Pip is a virtual pet that introduces users to reinforcement learning through direct interaction.

The idea is simple. You start with a small creature that doesn't already know how to behave. You interact with it, give it feedback, and gradually influence what it learns to do.

Unlike a traditional virtual pet, where pressing a button triggers a predetermined response, Pip is designed to learn which actions are worth repeating based on its experiences.

Users don't need any background in machine learning to interact with Pip. They can experiment with rewarding different behaviors, observe how Pip responds, and see how its behavior changes over time.

I also want Pip to feel like a creature rather than a machine learning demonstration. It should be able to explore, make mistakes, and occasionally do things the user didn't expect.

The main goal is to let people experience how reinforcement learning works without having to understand the mathematics behind it first.

How we built it

We started by building a prototype that combines a virtual pet interface with the basic components needed for a reinforcement learning environment.

One of the first decisions was to separate Pip's behavior from its appearance. The visual interface is responsible for showing what Pip does, while the agent's learning system is intended to determine which actions it chooses.

We designed the system around a few basic components: the environment Pip interacts with, the observations it receives, the actions it can perform, and the rewards that influence its behavior.

We also wanted to avoid using a pretrained language model to generate Pip's personality or responses. The whole point of the project is to explore what an agent can learn from scratch, so using an already intelligent model would defeat much of that purpose.

The current prototype gives us a starting point for developing and testing these ideas.

Challenges we ran into

The hardest part was figuring out what Pip should actually learn.

In most reinforcement learning projects, the objective is straightforward. An agent might need to reach a destination, win a game, or avoid obstacles. But Pip isn't supposed to solve one specific problem. It's supposed to behave like a creature that learns from its surroundings.

That makes designing the reward system much more complicated. How do you encourage curiosity? How do you make sure Pip learns from feedback without simply repeating the same rewarded action forever?

Another challenge was making learning visible. An RL agent can update its policy internally, but that doesn't necessarily mean the user will notice anything different. Since the purpose of Pip is to help people understand RL, being able to observe meaningful behavioral changes is essential.

We also struggled with finding the right balance between scripted behavior and actual learning. Adding predefined animations and reactions makes Pip feel more alive immediately, but relying too heavily on them would undermine the entire idea.

Accomplishments that we're proud of

I'm proud that we managed to turn an idea into an initial working prototype.

What started as a question about virtual pets gradually became a much more interesting project about how people interact with learning agents.

I'm also happy with the direction we chose. It would have been much easier to create a cute character powered by a chatbot and call it an AI pet. Instead, we wanted to investigate how a creature could develop behaviors from its own experiences.

There's still a lot to improve, but having a prototype gives us something concrete to experiment with.

What we learned

One of the biggest things I learned is that reinforcement learning is as much about designing the environment as it is about the algorithm itself.

An agent can only learn from the information it receives, the actions available to it, and the feedback it gets. Even small decisions about these components can completely change its behavior.

I also realized that building an agent that learns and building an experience that helps people understand that learning are two different challenges.

For Pip, it's not enough for the algorithm to work. Users need to feel that their interactions matter and be able to recognize how those interactions affect Pip's behavior.

This project also made me appreciate how much work goes into making even a simple virtual creature feel responsive and engaging.

What's next for Pip

There's still a long way to go before Pip becomes the creature I originally imagined.

My next priority is improving its learning system and making behavioral changes easier to observe. I want users to be able to experiment with different rewards and see how those decisions affect Pip over time.

I also want to give Pip a more interesting body and environment. Eventually, I'd love for it to live directly on the desktop, moving around windows, exploring, and reacting to the user's activities.

Another goal is to make each Pip develop differently depending on how it is raised. Two users could start with identical agents but end up with very different pets because of the interactions and feedback they provide.

Ultimately, I want Pip to be something people can learn from without feeling like they're studying.

You start by trying to teach a little creature how to behave. Along the way, you begin to understand how reinforcement learning actually works.

Built With

Share this project:

Updates

Submission history