AI Research Hallucinations, Starting with Hormones and Tears

Hi there, it's Steph 👋

Last week, AI told us that women in our study were hormonal and crying due to stress. Here’s the kicker: We never asked those questions.

AI completely made it up. And oof, that one hit a nerve, not just because it was wrong, but because it leaned hard into tired, sexist stereotypes. I say that fully aware of my own blind spots as someone leading an all-women team.

This isn’t an anti-AI rant. We use it daily (and quite love it). It helps us move faster, spot patterns, and scale our output. On a good day, it triples what a single researcher can do.

But it’s helpful… until it starts confidently making stuff up.

In the past year, I’ve seen AI fabricate quotes, invent stats, and spin insights that sound smart but aren’t rooted in the data. They feel true, which makes them risky. No one wants to bet their roadmap on a hallucination.

We know most teams are already using AI to expand their thinking, from desk research to internal studies. We do too. So in the spirit of helping you get the most out of it (without getting tripped up), we’re sharing 5 red flags to watch for when using AI in research.

Because when AI hallucinates, it doesn’t come with a warning label (yet).Best,Steph

‍

TL;DR: AI can sharpen research, or sabotage it.

Watch for these 5 red flags to keep your insights human, honest, and hallucination-free: 1) phantom quotes, 2) made-up stats, 3) overconfident insights, 4) missing tension, and 5) bias reinforcement.

‍

Field Notes

Alright, let’s talk about how to actually get the best out of AI when you're running your own research.It can be a game-changer (faster, lighter, more output), but only if you know what to watch for. Otherwise, things start slipping through the cracks, and suddenly your “insights” don’t pass the sniff test.

Here are 5 red flags we keep an eye on every day:

1. Phantom quotesAsk AI to pull verbatims for color, and sometimes it just… makes them up. Like, totally fabricated quotes that sound great, but never actually appeared in your raw data. It’s wild. Always double-check! Your participants and respondents deserve to be represented accurately, as the humans they actually are.
2. Made-up statsAI doesn’t like gaps, so it fills them. Need a number? Boom! It might invent one. We’ve seen it throw out sample sizes, percent shifts, even rankings that don’t exist. Our team does what we call an “Eagle Eye” review to catch this. Incorporate a version of the Eagle Eye into your QA stage as a best practice.
3. Overconfident insightsThe algorithms love connecting dots, sometimes too fast. They’ll take a loose hunch and spin it into a headline-worthy takeaway, no warning included. If you read something and think, “Ooh, that’s juicy!” or “Whoa, that’s bold!,” pause. There’s a good chance it’s overreaching just to provoke that exact reaction.
4. Missing tensionSome of the best insights live in contradiction. Those stop-mid-scroll hesitations, the “yes, and…,” the messy stuff... AI tends to smooth that out. So zoom out of your analysis and review the raw data yourself. What isn’t being said? If I were a betting woman, I’d say AI always misses something important.
5. Bias reinforcementAI is trained on human patterns, which means human bias is baked in. If something sounds like a stereotype (hello crying women stat), triple-check it. It might feel true because you’ve heard it before, not because the data backs it up. Don’t let a lazy algorithm sneak in outdated thinking.

We love research. We love the people doing it. And, when it’s done well, research makes the world better for real humans, not just businesses that serve them.

In that spirit, we hope this guide helps you keep your research sharp, your insights honest, and your AI on a short leash. 😉