PersonaWorld

Co-work Zoom Chat
PersonaWorld User Profile selection
PersonaWorld Home Page

Inspiration

Teams building agentic products routinely make decisions about large user populations from evidence gathered from only a few people. PersonaWorld makes that gap testable. It attaches a structured, inspectable persona to an agent that can already use a browser, converse with an application, complete a questionnaire, or operate a desktop workflow.

A prompt that says “act like a 34-year-old nurse” is role-play. It does not provide a reproducible cohort, a sampling strategy, or evidence that the persona changed the decision. PersonaWorld treats a user study as an execution artifact: a persona, a task, an agent, a runtime, a submission, and a verifier that can be inspected and rerun.

What it does

PersonaWorld pairs synthetic personas with LLM agents inside controlled task environments and evaluates the resulting behavior. The atomic unit is a trial: one persona acting on one task under a fixed model, agent, and execution configuration. A job can replicate that trial across a seeded cohort, making a product question measurable instead of anecdotal.

The platform supports four scenario families under one contract:

persona → agent → runtime → submission → verifier → result

Survey — stated preferences, pricing reactions, and product feedback.
Chatbot — multi-turn support or recommendation conversations.
Web — browser research, comparison, and preference-sensitive choices.
OS app — end-to-end workflows such as reviewing an iOS photo permission.

All four surfaces emit the same evaluation artifact. The schema contains 1,290 dimensions and 6,347 values across 43 categories, and a cohort audit runs before any expensive job, halting continuous integration when a population is invalid or insufficiently covered.

How we built it

Attributes were harvested from existing instruments rather than authored: ACS PUMS, the GSS codebook, World Values Survey Wave 7, HEXACO-PI-R, the IPIP item pool, Primal World Beliefs, BIS/BAS, and persona taxonomies including DeepPersona and SCOPE-Persona. A six-stage pipeline aggregates, normalizes, and categorizes candidates, then applies embedding retrieval with LLM adjudication to collapse near-duplicates that string matching misses.

Because the reachable space is the product of all value counts, approximately $10^{885}$, independent sampling per dimension yields incoherent individuals. The schema is instead interpreted as a DAG-structured proposal distribution and sampled in topological order. For node $i$,

$$q_i(v) \;\propto\; P^0_i(v)\,\exp!\Big(\gamma_i\Big[\sum_j w_{ij}\,\ell_{ij}(v \mid x_j) + \sum_c \lambda_c\,\ell_c\big(v \mid \mathbf{x}_{\mathrm{pa}(c)}\big)\Big]\Big)\prod_m \mu_m(v),$$

where $P^0_i$ is the node prior, $\ell_{ij}$ and $\ell_c$ are pairwise and full-table log-likelihood ratios against it, $\mu_m$ are conditional masks, and $\gamma_i$ is per-node shrinkage on the accumulated evidence. Compilation folds every weight into float32 lookup tables, and nodes are drawn by unnormalized inverse-CDF sampling, which avoids an explicit normalization pass.

In parallel, a pipeline extracts persona attributes from records of real people. It is sized for a corpus on the order of a million Wikipedia-derived profiles, together with a selected index of one hundred thousand Amazon reviewers, and runs Qwen3.6-35B-A3B under vLLM across 200 resumable SLURM shards on H200 GPUs. Every field carries a value, a confidence, a verbatim quotation, and an assignment type, permitting downstream filtering to what the source actually supports. Execution builds on Harbor, which ingests Terminal-Bench tasks through a mapper, extended with a persona layer that renders the persona into the system prompt and uploads it into the container so in-container verifiers score against the persona itself. Grading combines deterministic file, JSON-path, SQLite, HTTP, and trajectory checks with LLM outcome and trajectory judges.

Challenges we ran into

The central difficulty was representing correlation without specifying a joint distribution over 1,290 categorical variables. Obtaining realistic dependence from pairwise edges, selective conditional probability tables, and local masks, without collapsing an entire cohort onto one archetype, is the specific purpose of the shrinkage coefficient $\gamma_i$; diagnosing that failure required building the audit tooling before the sampler could be tuned. Speed was a second constraint, and precompiled lookup tables, unnormalized inverse-CDF sampling, and nibble-packed codes reduced cohort generation from minutes to seconds.

The most consequential problem was demonstrating that the persona has any effect, since an agent may read a detailed persona and still respond generically. Each grounded task therefore fixes its confounders, including income band, age bracket, and risk tolerance, probes exactly one dimension, and scores responses against an oracle whitelist with an alignment threshold of 80 percent. Beyond that, reconciling four dissimilar execution surfaces under one contract required three revisions of the artifact schema, and large-scale extraction had to be made shard-resumable throughout to survive preemption and partition time limits.

Accomplishments that we're proud of

Every attribute in the 1,290-dimension schema traces to a published instrument rather than to internal invention, and the sampler draws internally consistent personas from a $10^{885}$-configuration space fast enough to sit behind an interactive playground. Four scenario families run against functioning applications through a single verifier contract, which makes cross-family comparison meaningful rather than anecdotal. The result we regard most highly is the validity protocol, which converts an impression that the persona mattered into a quantity capable of returning a negative finding.

What we learned

Persona conditioning requires substantially more infrastructure than prompt authoring. Once coverage, stratification, and reproducibility are required, the prompt accounts for only the final stage of the pipeline. Simulation without a validity test provides no evidence, and knowing in advance that a dimension must be shown to move behavior changed how tasks were subsequently written. The artifact contract proved to be the load-bearing design decision, since identically shaped evidence made judging, aggregation, and reporting follow at low marginal cost.

What's next for PersonaWorld

The immediate priority is calibration: fitting node priors and edge weights so that cohorts reproduce known ACS, GSS, and WVS marginals, with a calibration report accompanying each released dataset. The subsequent objective is a validation study administering the same instrument to real respondents and to a matched synthetic cohort, reported on divergence rather than agreement alone. Further work includes releasing the extraction corpus and persona graph, opening the playground interface to non-engineers, and making sweeps first-class so that a single cohort can be run across models to separate the contribution of the persona from that of the underlying model.