Inspiration
Founders often receive plenty of positive feedback and very little reliable evidence.
People say an idea sounds useful. Colleagues praise the concept. Early users express interest. Yet when the conversation turns to budget, authority, alternatives or an actual buying decision, the apparent demand can disappear.
We wanted to explore whether AI could help expose that gap earlier.
The idea behind A Thousand Interviews Overnight was to create a synthetic research panel that does not behave like an enthusiastic brainstorming partner. Instead, it should behave like a sceptical market-research process: asking structured questions, challenging assumptions and distinguishing interest from credible willingness to pay.
The goal is not to replace real customer discovery. It is to help founders develop sharper hypotheses before they spend weeks recruiting participants, conducting interviews and analysing transcripts.
What it does
A Thousand Interviews Overnight runs structured synthetic customer interviews against a proposed product, target market and pricing hypothesis.
The system:
Creates a weighted panel of synthetic respondents with different roles, company sizes, geographies, pain levels and purchasing authority. Interviews each respondent separately using a fixed research guide. Deliberately looks for rejection, indifference, weak authority, credible alternatives and pricing resistance. Produces structured outputs for pain intensity, budget authority, purchase verdict, objections, must-have status and price thresholds. Groups respondents into evidence-based segments. Calculates a payer-evidence score for each segment. Produces Van Westendorp-style price analysis. Displays the results through an interactive dashboard with segment rankings, objections, pricing ranges, respondent maps, quotes and full transcripts.
The dashboard is designed to answer a more useful question than “Did people like the idea?”
It asks:
Who showed the strongest evidence that they might actually pay, and why?
Every result is explicitly presented as synthetic and directional. The output is intended to improve the next round of real-world customer research, not to create a false sales forecast.
How we built it
The project combines Codex, GPT-5.6, TypeScript and deterministic local analytics.
A local command-line runner manages the full research process:
The user supplies a product description, target customer hypothesis, pricing hypothesis and panel requirements. GPT-5.6 generates a weighted respondent roster. Codex runs each synthetic interview as an isolated call. Each response is validated against strict JSON schemas. Results are checkpointed after every completed interview. Deterministic analysis runs locally across the structured outputs. GPT-5.6 creates evidence-grounded segment names and summaries. Sanitised study data is published to a static Next.js dashboard.
The interview prompt is deliberately anti-sycophantic. Respondents are instructed not to improve, rescue or sell the product idea. They must answer from their assigned circumstances, existing behaviour, alternatives, authority and budget.
The interview guide uses seven consistent sections so that outputs can be compared across respondents:
Current behaviour Problem severity Existing alternatives Buying process Pricing Objections Final verdict
The analysis layer uses structured variables rather than embeddings. It applies standardisation, deterministic k-means clustering, silhouette-based cluster selection, principal component analysis and Van Westendorp-style pricing intersections.
The final viewer is a static application. It requires no hosted model credentials and does not generate new interviews in the browser.
Challenges we ran into
The hardest challenge was preventing synthetic respondents from becoming helpful assistants.
Large language models naturally tend to agree, elaborate and improve ideas. That behaviour is useful in many contexts, but harmful in customer research. We had to design prompts that encouraged respondents to reject weak propositions, expose uncertainty and remain grounded in their assigned role and authority.
A second challenge was turning qualitative interviews into comparable evidence without pretending the output was more precise than it really was. We addressed this by enforcing a strict schema, separating generated interpretation from deterministic analysis and labelling the final score as payer evidence rather than purchase probability.
Long-running execution also required careful engineering. A panel may involve dozens or hundreds of separate model calls, so the runner needed:
Per-interview checkpoints Resume support Call-budget enforcement Schema validation Rate-limit handling Clear diagnostics Safe interruption Reproducible downstream analysis
We also encountered a Windows-specific issue when launching the packaged Codex executable. The runner required a PowerShell wrapper rather than directly spawning the executable with PowerShell arguments. We corrected the launcher, extracted the behaviour into a testable function and added regression coverage and continuous integration.
The final challenge was balancing transparency with presentation. We wanted the dashboard to feel compelling while making it impossible to mistake synthetic evidence for real customer validation.
Accomplishments that we're proud of
We are particularly proud that the system is willing to produce commercially uncomfortable results.
The two demonstration studies each interviewed 50 synthetic respondents. Neither produced a single unqualified “buy” verdict. Most respondents preferred a trial, delayed the decision or rejected the proposition.
That may appear disappointing, but it is exactly the behaviour the system was designed to produce. A research tool should not reward the founder with artificial enthusiasm. It should expose weak evidence before the founder mistakes interest for demand.
We are also proud of the separation between model-generated work and deterministic analysis.
GPT-5.6 handles interpretation-heavy tasks such as panel creation, interviewing and evidence-grounded segment naming. Clustering, scoring, pricing analysis and visualisation are performed in ordinary application code. This makes the process easier to inspect, test and explain.
Other accomplishments include:
A complete local interview pipeline. Strict schema validation for every response. Per-interview checkpointing and resumability. An interactive evidence dashboard. Full transcript inspection. Exportable Markdown reports. Explicit methodological warnings. No requirement for hosted credentials in the public viewer. Automated tests, type checking, static build verification and CI.
What we learned
The largest lesson was that prompt tone alone is not enough to produce credible synthetic research.
A useful process needs constraints, structure and evidence requirements. Respondents need specific circumstances, limits and alternatives. Questions need to test current behaviour rather than hypothetical enthusiasm. Outputs need bounded fields that can be validated and compared.
We also learned that synthetic research becomes more useful when it is treated as a hypothesis-generation tool rather than a substitute for customers.
The strongest output is not a definitive answer. It is a better next question:
Which segment should be interviewed first? Which objection should be tested? Which pricing assumption looks weakest? Who has pain but no purchasing authority? Which alternative is the real competitor? What evidence would change a “not now” into a trial?
Finally, we learned that negative results are a feature.
If every synthetic respondent loves the product, the panel has failed.
What's next for A Thousand Interviews Overnight
The next stage is to compare synthetic findings with real customer interviews.
We want to measure where synthetic respondents are directionally useful, where they systematically differ from real buyers and how the scoring model should be calibrated.
Planned improvements include:
Side-by-side comparison of synthetic and real interviews. Calibration against real purchasing outcomes. Richer panel weighting and respondent controls. Industry-specific interview frameworks. Improved pricing-confidence measures. Better explanation of cluster and PCA drivers. Stronger accessibility across the dashboard. Private team workspaces and controlled study publishing. Integrations with product-discovery and research tools. Automated generation of real-world validation plans.
The long-term vision is not to automate customer discovery.
It is to help teams arrive at real customer conversations better prepared, less biased and more willing to hear that enthusiasm is not revenue.
Log in or sign up for Devpost to join the conversation.