When synthetic respondents tell you exactly what you want to hear

There's a conversation that's been swirling around nearly every research meeting I've been in lately. You know that feeling when everyone's whispering about the same thing but nobody wants to say it out loud? That's where we are with synthetic samples. I'm real-time wondering... are we about to replace the very thing that makes our work human?

The promise is intoxicating, honestly. Picture this: no more begging people to take surveys, filtering out bots/garbage responses, no more waiting weeks for data, no more privacy lawyer meetings that make your soul die a little. Just clean, fast, AI-generated responses that supposedly think and feel like real consumers.

But here's what I'm seeing behind the curtain: the models we're so eager to trust are basically that friend who only talks about things that happened six months ago. They're missing the cultural moments, the trending anxieties, the weird little shifts that somehow change everything. You also have a truth problem wrapped in statistical significance when the data contradicts expectations.

For now, we're not jumping on this bandwagon here at Two Cents. But, we are going to be doing the unglamorous work of actually testing things because I refuse to put our name on insights we can't defend.

The real question isn't whether synthetic samples will eventually work......It's what we're willing to call “truth,” and how much proof we need before we tell a business to bet their future on it.

‍

🚀 UPCOMING NEW SERIES ALERT: Research leaders, unfiltered

Our new Research Fellow ​Manasi Chandu​ is taking over the next few editions of our newsletter to do what we do best: get people to say the quiet parts out loud.

She's sitting down with Heads of Research from companies you definitely know to talk about the stuff that doesn't make it into the thought leadership posts; how they actually understand users, what research team architecting really looks, and their hot take on the best (and worst) frameworks and methods to use.

‍

TL;DR: Synthetic data isn't just a cool new tool.

It's forcing us to confront what "real" consumer insights actually mean in the first place. We're at the beginning stages of our discipline fundamentally changing.

‍

Field Notes

A Guide to Synthetic Data (without vendor BS)

Here’s what I’m seeing as teams test synthetic respondents...

The promise

• Thousands of “respondents” in hours, not weeks
• Concept tests without recruiting or privacy headaches
• Scenario modeling that is impossible with real people
• Instant fills for hard-to-reach audiences, globally

The reality checkEarly tests are mixed. In one researcher group I'm in, an expert champions the accuracy of the models and their produced results, and in that same group another said, “There is no evidence to support the validity of a synthetic sample or synthetic panelist that I have seen. And I would challenge any such evidence with an extreme degree of skepticism.”

What people say is working

• Stress-testing concepts before real fieldwork
• Exploring “what if” scenarios like demographic or economic shifts
• Augmenting small samples in niche categories
• Fast, directional reads when speed matters

What’s making me nervous

• Models learn from the past to predict the future, which can echo yesterday
• When synthetic conflicts with human data, what is the source of truth
• Bias can amplify and masquerade as significance
• Clean outputs can hide the real-world messiness that actually drives behavior
• We know for a fact that AI wildly hallucinates today

The uncomfortable truthThis is less about “is it accurate” and more about “what are we trying to learn?”Quick, cheap assumption checks: synthetic might do the trick (as does ChatGPT).Understanding surprising, emerging, contradictory human behavior: jury still out.

Three-move-ahead strategy

• Start small: run parallel studies on low-stakes work
• Deeply question results, especially when they confirm your assumptions
• Be ready to answer executives on how much you trust the results, and why

The best researchers I know are not asking whether to use synthetic, they’re trying to figure out when, how, and what it means for our craft.

What’s your take?