Product Oracle

• 

  Hook

Inspiration

Product teams often build features based on assumptions, limited user interviews, and feedback from a small set of customers. While traditional research is valuable, it is slow, expensive, and difficult to repeat every time a product changes.

We wanted to explore a new approach: what if product teams could create realistic AI personas representing their target users, let those personas interact with a product, and observe where they struggle before releasing it to real customers?

That led us to build Product Oracle—an AI-powered product testing platform that simulates how different users may experience a product and turns those interactions into actionable insights.

What it does

Product Oracle allows teams to provide a product, prototype, website, or application along with information about their target users.

The platform then:

1. Generates multiple AI personas based on the product’s intended audience.
2. Assigns each persona realistic goals, behaviours, technical abilities, expectations, and frustrations.
3. Lets these AI agents explore and interact with the product independently.
4. Records the actions, decisions, confusion points, and failures experienced by each persona.
5. Converts the sessions into a visual product research report.

The final dashboard highlights:

• Common usability problems
• Drop-off and hesitation points
• Tasks that users fail to complete
• Differences in behaviour across personas
• Confusing copy or navigation
• Missing features and unmet expectations
• Severity and frequency of each issue
• Recommended product improvements
• A prioritised list of fixes based on expected user impact

For example, a founder testing a financial application could simulate a first-time investor, an experienced trader, a cautious parent, and a non-technical user. Each persona would approach the product differently, helping the team identify problems that may not appear during internal testing.

Product Oracle is not intended to replace real users. It acts as an early product-testing layer that helps teams discover obvious and high-impact problems before spending time and money recruiting participants or launching the product.

How we built it

We built Product Oracle as a multi-agent product simulation system.

The workflow begins when the user submits a product URL and describes the target customer, the product’s purpose, and the tasks they want to test.

The system then analyses the provided context and generates a diverse group of AI personas. Each persona is given its own:

• Background
• Level of product knowledge
• Motivation
• Primary objective
• Behavioural traits
• Technical proficiency
• Concerns and objections
• Success criteria

Each persona is converted into an autonomous agent capable of navigating the product and attempting assigned tasks.

As the agents interact with the product, the system captures their actions, reasoning, task progress, errors, hesitation points, and final outcomes. These individual sessions are then passed through an analysis layer that identifies repeated patterns across the personas.

The platform combines these findings into a visual dashboard containing persona cards, session timelines, issue clusters, severity scores, product recommendations, and an overall product-readiness assessment.

We designed the system around structured agent outputs so that the findings could be compared across sessions instead of producing only unstructured AI feedback.

Challenges we ran into

One of the biggest challenges was making the personas meaningfully different. Without sufficient behavioural constraints, multiple agents would interact with the product in nearly identical ways.

We addressed this by giving each persona distinct motivations, experience levels, expectations, objections, and definitions of success.

Another challenge was preventing agents from behaving like product experts. AI models often infer how an interface is supposed to work, even when an actual user might be confused. We had to encourage the agents to operate only using information visible inside the product and avoid making assumptions based on hidden knowledge.

We also had to distinguish between genuine usability problems and failures caused by the simulation itself. To solve this, we looked for repeated issues across multiple personas and gave higher confidence to findings that appeared consistently.

Turning long agent sessions into useful product recommendations was another major challenge. Raw session transcripts are difficult for teams to act on, so we developed an analysis layer that clusters related problems, estimates severity, identifies affected personas, and recommends specific fixes.

Finally, building a system that could work across different types of products required us to avoid hard-coding the experience around one application or industry.

Accomplishments that we’re proud of

We are proud that Product Oracle goes beyond generating generic AI feedback.

Instead of asking an AI model, “What is wrong with this product?”, the platform creates multiple users, gives them goals, lets them experience the product, and bases its findings on their behaviour.

We successfully built a workflow that transforms:

Product context → AI personas → simulated product sessions → issue detection → prioritised recommendations

We are also proud of the visual experience. Product teams can see each persona, follow their journey, understand where they struggled, and compare behaviour across different user types.

Another key accomplishment was making the output actionable. Product Oracle does not only say that a screen is confusing. It explains:

• Who was affected
• What they were trying to achieve
• Where they became stuck
• Why the problem matters
• How frequently it occurred
• What the team should change

This makes the platform useful not only as a demonstration, but as a potential part of a real product-development workflow.

What we learned

We learned that AI personas become significantly more useful when they are treated as behavioural systems rather than simple demographic profiles.

A persona’s age or job title alone does not meaningfully change how it uses a product. Its motivations, familiarity, urgency, risk tolerance, expectations, and technical ability have a much greater impact.

We also learned that individual AI feedback should not automatically be treated as truth. The strongest insights come from patterns across multiple independent sessions.

Another important lesson was that simulation is most valuable at the earliest stages of product development. It can help teams test prototypes, onboarding flows, landing pages, feature concepts, and navigation before investing in large-scale development or user research.

Most importantly, we learned that AI can become more than a tool that generates ideas. It can act as a product-testing environment in which teams can run experiments, observe behaviour, and improve their decisions.

What’s next for Product Oracle

The next step is to make Product Oracle a continuous product-intelligence layer for product, design, and engineering teams.

We plan to add:

• Automated browser-based interaction with live products
• Support for Figma prototypes and staging environments
• Screen recordings and visual session replays
• Persona generation from real customer research and analytics
• Integration with tools such as Jira, Linear, Slack, and Notion
• Automatic ticket creation for critical issues
• Comparison of product performance across different versions
• Regression testing using the same personas after every product update
• Accessibility and localisation personas
• Industry-specific persona libraries
• Confidence scores based on repeated agent behaviour
• Human-feedback calibration using real usability-testing results

In the future, teams could connect Product Oracle to every product release. AI personas would continuously test important workflows, identify regressions, and notify teams when an experience becomes more difficult for a particular customer segment.

Our long-term vision is to create a product simulation environment where teams can test not only whether their software technically works, but whether different types of people can successfully use it.

Before shipping to thousands of users, teams should be able to ask Product Oracle:

“How will my customers experience this product?”

Built With

• claudecode
• codex
• firecrawl
• novus
• openai