PersonaAgent

Inspiration

A website can pass every scripted test and still be difficult for people to understand.

Traditional automated tests are valuable for checking whether a known path works. They can confirm that a page loads, a button exists, or a form submits successfully. However, they usually assume that the person using the product already understands the interface and knows where to begin.

Human usability studies provide deeper insight, but they can be expensive, slow to organize, and difficult to repeat after every design change.

We built PersonaAgent to give product teams a faster and more repeatable way to investigate usability. It provides evidence of how different user perspectives approach the same interface, where they become uncertain, and which parts of the experience need attention.

What PersonaAgent Does

PersonaAgent turns a website into an AI-assisted usability testing environment.

A user provides a website URL, describes a task, and selects one or more personas. Each persona receives its own browser session and attempts to complete the task according to its goals, experience level, and constraints.

The agent can navigate pages, click controls, enter information, search, scroll, inspect content, and evaluate whether the task was completed successfully. Every action can include the agent’s reasoning, its outcome, the current page state, and a screenshot.

When the run is complete, PersonaAgent produces a summary, an experience score, a written score justification, an action history, screenshots, and persona-specific findings. This allows teams to compare how different perspectives understand the same product and identify changes that could make the experience clearer and easier to use.

How We Built the Project

PersonaAgent uses a React and TypeScript frontend connected to a FastAPI backend. The frontend manages test setup, persona selection, run history, progress states, result review, screenshots, and proposed improvements. The backend coordinates persona execution, browser sessions, result validation, screenshot storage, and image-generation jobs.

Steel provides the remote browser sessions, while Playwright connects to those sessions and performs actions such as navigation, clicking, typing, scrolling, and reading page content. The AI model acts as the reasoning layer, using the persona’s goals, the assigned task, the current page state, and the previous action history to decide what to do next.

Each browser action produces structured evidence, including the selected action, the agent’s reasoning, the outcome, the page state, and a screenshot. When the task is complete, the backend returns the action history, completion result, experience score, and score justification. Completed screenshots can then be sent to an image-editing model to generate proposed improvements based on the observed findings.

The system supports multiple personas and custom persona definitions so teams can compare different user perspectives against the same website and task. During development, we first built a deterministic baseline to verify the core workflow, result states, and evidence display. Once that foundation was reliable, we connected the application to live Steel browser sessions and tested the complete end-to-end flow.

What We Learned

The biggest lesson from building PersonaAgent was that an AI testing agent is a systems problem, not only a prompting problem. A useful agent needs a clear objective, a structured persona, reliable browser tools, explicit session management, and a way to verify whether the task was actually completed.

We learned that convincing reasoning is not enough. An agent can explain its decisions clearly while still choosing the wrong page, repeating an action, or failing to complete the user’s goal. PersonaAgent addresses this by connecting every result to observable browser actions, screenshots, outcomes, and a final score justification.

We also learned that personas should change behavior rather than simply change the tone of a response. A first-time visitor, an experienced researcher, an older adult, a mobile user, an accessibility advocate, and a privacy-conscious visitor should inspect different parts of an interface and encounter different usability concerns.

Building a deterministic baseline before relying entirely on live AI and browser services taught us the value of a stable foundation. It allowed us to validate the frontend, persona journeys, screenshots, progress states, and comparison interface independently from model availability and remote browser behavior.

The project also showed us how important session lifecycle management is. Browser automation involves several connected systems, and a failure in one layer can appear as an unrelated problem elsewhere. Explicitly managing browser connections, remote sessions, viewers, and stored evidence made the overall workflow more reliable.

Finally, we learned that generated redesigns are only useful when they remain grounded in evidence. A polished image is not valuable by itself. It becomes useful when a reviewer can trace it back to a specific persona, action, screenshot, complaint, score, and justification. This connection between observation and proposed improvement is what makes PersonaAgent more than a generic AI website review.

Challenges We Faced

One of the hardest parts of building PersonaAgent was coordinating several systems at once. The frontend, API server, AI reasoning layer, Playwright automation, Steel browser sessions, screenshot storage, and background jobs all had to remain synchronized throughout a test run.

We encountered a lifecycle issue where a Steel browser session was released while its viewer was still mounted in the frontend. This caused the interface to display “Browser Disconnected” even though the browser had completed its work correctly. We resolved this by making the lifecycle explicit: Playwright disconnects first, the remote session is released afterward, and the frontend removes the viewer while preserving the completed results and screenshots.

Agent reliability created another challenge. An AI agent can produce convincing reasoning without completing the intended task. To reduce this gap, we added clear task objectives, structured persona information, observable browser actions, deterministic progress checks, maximum step limits, explicit action outcomes, and completion validation.

We also wanted the reasoning layer to remain independent from browser control. By separating model decisions from Playwright and Steel operations, the system can support provider fallback and future model changes without rewriting the core browser workflow. This separation also made it easier to identify whether a failure came from the model, the browser session, or the application itself.

What's Next

Our next steps are to improve persona creation, support longer and more complex workflows, make results easier to compare, and provide clearer summaries of recurring blockers and usability issues. We also plan to support larger test suites in which many personas can evaluate a product automatically, identify patterns across journeys, and show which problems consistently affect different users.