PartyRock Village

Inspiration: An AI Virtual Town Experiment by Stanford and Google Researchers to Study Human Behaviour

On April 7 2023, a team of AI researchers at Google and Stanford University posted a study online where they populating a virtual town with artificial intelligences and observing their behavior. They create 25 “generative AI powered agents,” or unique personas with identities and goals, and placed them into a sandbox environment resembling a town called Smallville much like The Sims (a series of life simulation video games). The authors of the study observed the agents as they went about their days, going to work, talking and interacting with one another, involves in different events, and even starting to plan activities. Each AI agent is actually an instances of Open AI’s ChatGPT, embodies a character in the fictional town. For example, John Lin, a helpful pharmacy shopkeeper with a loving family and friendly neighbors. Prompted with specific scenarios, like waking up in the morning, these AI characters autonomously decide their next actions. However, beneath the surface, it's not a virtual reality simulation but rather a sophisticated text-based interaction. Interactions occur through prompts and responses, with each AI unaware of others' actions until informed by the experimental framework. In simpler terms, the experiment makes it seem like the AI characters, such as John talking to his son Eddy, are doing their own thing, but actually, everything is carefully planned out. Even though it's complicated, the way they talk and interact seems real, like in a choose-your-own-adventure story. Users can introduce new elements like a dripping faucet, and the AI characters respond accordingly, treating any input as reality. Yet, maintaining coherence amidst vast amounts of detail requires constant oversight and reminders. Although far from perfect, this experiment showcases the potential of large language models like ChatGPT in simulating human behavior. While initially unforeseen, this capability hints at broader applications beyond gaming and virtual environments, as well as promising a more sophisticated simulations of human interaction in diverse contexts.

My Conclusion:

🔥Therefore, upon reviewing the research paper detailing this experiment and discovering the AWS PartyRock platform during the hackathon, I am inspired to replicate a similar simulation on PartyRock. Given that I have access to the necessary AI widgets for setting up the experiment, such as text generation and chatbot widgets, I can explore various models and fine-tune them using a user-friendly interface. Additionally, once completed, I can effortlessly share this simulation with the public.

How I recreate the simulation on AWS PartyRock

Step 1: Create the simulation agents (villagers)

In order to develop a simulation agent for PartyRock Village, I constructed a comprehensive background story for each agent, including details such as identity, occupation, workplace, habits, daily routine, and interpersonal connections. This background narrative served as the basis for generating a description for each simulation agent, providing a reference point for the primary AI model used in the experimental setup. Initially, I planned to utilize a text generation model within another PartyRock project to facilitate the creation of characters and their respective backgrounds. However, I discovered that the relationships and connections between agents lacked coherence, as they were generated independently in each run of the widget. Consequently, I decided to manually design all the agents and craft individual background stories for them, ensuring that the relationships between them were logical and interconnected, thus enabling the introduction of meaningful connections and interactions during the experiment. Here’s an example prompt of an agent I created, named Jimmy:

Jimmy is a full stack developer which works at the Spider Technology. He study Computer Science and graduated from the University of PartyRock. Jimmy is a hardworking person and never being late to work, he always will be first one arrived at the office. He loves to drink coffee and always will drop by Mina’s Coffee to have an Americano before heading to his office. Jimmy lives in a rental apartment near his workplace and will only go home and meet his parent, James and Michelle during the weekend. Jimmy do not like home as he prefer staying alone after a hectic working week, but Michelle always urge him to come back home frequently, like what usual mum will do. Jimmy has two besties which lives in the same neighborhood, Julie and John Samson. Jimmy always hangout with them in the park and have chill chit-chat session. Jimmy’s only hobby is to watch movie, so whenever he knows there is a new movie coming out, he will invite his two besties to join him. During his free time, Jimmy will help his parents at their family grocery shop.

Finally, I had created a total of 10 simulation agents, below is a quick summary of all of the agents (which the avatars are generated by a simple PartyRock app I built ❤️):

After having all the background story of the agents, I had created a text generation widget for each of the agents to generate their character description based on my designed prompt. All of the text generation widget is powered by the Claude model, with 0 temperature and Top P (to avoid the randomness and creativeness in this stage, as I need to ensure the character description will be close to what I had designed). To make the simulation interface clean, all of the agent widgets are under the villagers controller section:

Step 2: Create the locations in the virtual village

To enhance the complexity of the simulation and facilitate the emergence of interactions among agents, I introduced 10 distinct locations into the experiment setup to establish a virtual village, which is named The PartyRock Village. These 10 locations were designed in alignment with the characteristics of the agents, incorporating details such as the background information of the location, its specific role within the village, proximity to other locations, and the frequent presence of certain agents or ownership of the location. Here’s an example of a location I created, the Spider Technology Office:

• Background: A modern tech hub where Jimmy, the full stack developer, works diligently on various projects. The office is known for its innovative work culture and cutting-edge technology solutions.
• Location: Central Business District (CBD) of PartyRock Village.
• Distance from Mina's Coffee: 0.3 miles.
• Owner/Manager: The CEO of Spider Technology.

Finally, I had created a total of 10 locations for the PartyRock Village, below is a quick summary of all of the locations (similarly, the illustration of the locations are generated via a simple PartyRock app I built ❤️):

After having all the info for the 10 locations, I had created a text generation widget for each of the location to generate their location description based on my designed prompt. All of the text generation widget is also powered by the Claude model, with the same 0 temperature and Top P configurations (to avoid the randomness and creativeness in this stage, as I need to ensure the location description will be close to what I had designed). All of the location widgets are under the locations controller section:

Step 3: Setting up the simulation

The simulation is designed so that we first generate a story for the initial day in PartyRock Village, serving as the starting point for the entire simulation. The story of the first day is generated based on the setup of the agents and locations in steps 1 and 2. Therefore, I created a Claude-powered text generation widget named "Day 1 in the Village: Starting Point" to generate the story of the first day. This widget utilizes maximum temperature and Top P configuration to allow for maximum freedom and creativity in generating the story for this simulation. Once we have the starting point, we then store the story of the first day in a temporary memory called "What Happened to the Village Today?" created using the user input widget (the reason for which is mentioned in the challenges section below). Next, we proceed with the simulation for the following day.

Here's where the trick comes in: I created another text generation widget using the Claude model, named "The Next Day in the Village," also utilizing max temperature and Top P. This widget takes into account the setup of all the agents and locations from step 1 and 2, as well as the story of the previous day in the village. In simple terms, this widget continues the story of the second day using the temporary memory from the previous day. Therefore, for the next day, the user only needs to copy the output from the "The Next Day in the Village" widget to the temporary memory "What Happened to the Village Today?" and the simulation will continue (which "The Next Day in the Village" widget will update after the temporary memory is updated).

Additionally, to replicate the implementation of a dripping faucet in the Stanford and Google AI Research (where experimenters can manually introduce events to the simulation to observe the effects), similar to being the God of the virtual town where one can control everything, I introduced a chatbot widget powered by Claude named "GOD OF THE VILLAGE." This widget has access to the story of the day in PartyRock Village from "The Next Day in the Village" widget. As implied by the widget title, users can chat with the "God" (which is essentially an AI chatbot) to delve deeper into the story of that day in PartyRock Village, asking for more details about each event happening that day. It's worth mentioning that the prompt engineered for this chatbot not only can answer questions from the user related to the story in the village that day, but it will also sometimes reveal hidden details behind events and introduce curiosity to the user, leading them to ask more questions. This chatbot also utilizes maximum temperature and Top P configuration to introduce creativity and unexpected responses to some events.

Output analysis: A quick run through of the simulation I built

First, I started with generating the agents. The background description of the agents looks good and are following the prompt of respective text generation widget:

Next, I continue to generate the virtual locations in the PartyRock Village. As designed, the background description of each locations is well narrative and are following the prompt of respective text generation widget:

Now, I'm ready to run the simulation! First, I start with generating the story of Day 1. It looks good:

I pass the Day 1 story to the temporary memory widget to continue the story of Day 2. On Day 2, IT IS SUPER INTERESTING, as there are some new events emerge in the story. Plus, some of the events are a continuous of the events happening in Day 1. I had highlighted on the below screenshot of the output:

After having an interesting continuous story of Day 2, I want to know more details on some new events. Hence, I had asked two questions to the "GOD OF THE VILLAGE", since he knows everything! The answer replied from the chatbot GOD is comprehensive and detail, the creativeness and freedom in the answer is perfect:

Challenges for building this simulation

Problem 1: Memory on widget

The virtual village simulation requires the language model to have a memory of the previous days in able to generate a story of the next day, ensure a good continuous in the storyline. However, in PartyRock, the text generation widget has no memory. Every time the reference widget is updated, for example a new user input is provided, the text generation widget which has a reference to the user input widget will be flush and updated with a new response from the language model.

Solution: I create a user input widget to act as a temporary memory storage, so that the text generation widget which is responsible to generate the story of the second day will have a reference of what had happened in the previous day. However, this can only partially solve the memory challenges. This is because, sometimes, an event can be progress for a few days in the storyline, but since lack of memory (only have reference to the previous day), most of the events will only continue in a simplified way as the memory for the beginning of that event is not passed until "today".

Problem 2: Token limit for text generation widget

I had encountered token limit warning during the design of this PartyRock app, which is during generating the agent, locations and also story of each day.

For generating agent, the prompt of each agent is very detail, but when the background description of each agent is generated and pointed to the story generation widget to use it, the token limit is exceeded. So, there is only two solution for this. The first solution is to reduce the number of agents or redesign the prompt of the model to generate a simplified summary of the user (which the drawback is to sacrifice the detail of each agent background description). At last, I choose to remain the number of agents as it will create a more dynamic and interesting story, while redesign the prompt to shorten the background description of each agent. This works!

For generating the background description of each virtual location in the PartyRock Village, I also choose to remain the number of locations inside the simulated village, but choose to reduce the length of each location's description.

For story generation, I need to ensure that the story of the first day will not exceed the token limit. Plus, I also need to ensure that the story of the n th day will not exceed the token limit, so that I can pass to the temporary memory widget to continue the story on the n+1 th day. Hence, during the prompt engineering, I tell the language model to exclude the beautiful and bombastic description in the story, and making it much more concise and yet remain detail. Now, it performs perfectly.

Built With

• amazon-web-services
• claude
• partyrock