Inspiration
The academic behavioral science fields (psychology, sociology, economics, etc.) run studies with human participants. This is a massive industry with estimated $350 million spent each year on survey research. Unfortunately, a large portion of that money is wasted on failed experiments.
I kept hearing the same story from researchers: they would spend a few hundred dollars running a study online, wait for the data, and find out that the effect they hypothesized just wasn't there. That's a normal part of research, but it's a painful and expensive way to learn something. You're paying money and burning time to discover an idea doesn't hold up. I thought we might be able to pre-test some of these research ideas with LLMs instead of real people. LLMs are trained on how people talk and reason, so maybe they can mimic humans just enough to pre-test an idea for a fraction of the cost. That was the whole bet behind Doppelganger.
What it does
You upload your survey to Doppelganger and it runs the study for you using AI respondents instead of people. Each respondent is a distinct persona with its own background and response style, and they answer every question in character. We have developed a custom ensemble model that excels at mimicking humans. Because we submit through Qualtrics's Sessions API, the survey's real logic fires the same way it would for a human, including randomization and skip logic. A few minutes later you get back a full dataset and a summary of how the answers came out, so you can see whether your effect is showing up before you spend a dollar recruiting real participants.
How we built it
There are two layers. The core is a Python engine that reads a survey's structure, builds a persona for each respondent, and drives an LLM through the survey question by question. On top of that is a web app (FastAPI backend, Next.js frontend) with accounts, credits, and Stripe checkout, so anyone can run a study without touching code. Instead of trusting a single model, we run each study through an ensemble of several different LLMs and combine the results, since no one model is right about everything. Gemini runs on Vertex AI as one of those models. Gemini also provides an AI summary of the research results and recommended actions to improve the survey.
Challenges we ran into
The hardest part wasn't the AI, it was making the results trustworthy. It's easy to get an LLM to answer a survey; it's hard to know if those answers mean anything. We built a whole replication study around this, running 17 classic behavioral experiments through the models and checking how often they reproduced the real, published effects. The results of this work can be found in the paper we published - Do Large Language Models Reproduce Classic Human Decision Biases?. We also hit plenty of practical walls with the Qualtrics Sessions API, which rejects things like sliders and certain question types, so part of the work was figuring out what a survey needs to look like for AI to take it cleanly.
Accomplishments that we're proud of
We have paying customers! One customer was a repeat buyer which means that found value in the product and came back for more. Our ensemble reproduced about 91% of the known effects, which is well above what a single model manages. We turned that into a real product with paying customers. We also published our validation work on SSRN, because the only way researchers will trust this is if we're honest and specific about where it works and where it doesn't.
What we learned
In a skeptical field like academic research, evidence is the product. The paper, word of mouth, and the 17 study benchmark do more to earn a sale than traditional marketing. Getting LLM's to perfectly mimic humans and all their biases is tough work, but we can get close!
What's next for Doppelganger
Our long term plan is to earn trust in academic research first, then expand into the far larger private-sector market research and survey industry. We will continue to refine the simulations to more accurately align with real human survey takers and publish our results. The more we can prove our efficacy, the more customers we will win. Additionally, we will broaden the types of studies users can test on Doppelganger.
Log in or sign up for Devpost to join the conversation.