Garbage in, garbage out

Why real humans matter more than ever

One of the most exciting developments that I see in the application of AI in the research world is the fact that businesses and organisations are putting their research to good use and making sure it’s no longer sitting on a shelf collecting dust. Increasingly, brands are feeding years of reports, survey data, transcripts and videos into AI-powered knowledge systems, enabling teams to ask questions of it and maximise learnings in seconds. They can quickly check if they have already run research on a topic or specific audience, explore how sentiment has changed over time or quickly ask it the question that is keeping them awake at night. How fantastic is that.

As a practitioner, I’m delighted to know that my research will continue to add value to an organisation long after my project has been presented. Researchers have always talked about projects living beyond the debrief and it feels like these new tools and systems have finally solved one of research’s longest-standing challenges.

However, these systems assume something incredibly important: that the underlying data they rely on can be trusted. Of course, poor data isn’t a new phenomenon, however traditionally it only affects the individual project it relates to. Today the risk is far wider ranging and poor-quality research risks becoming embedded in organisational memory. We are no longer commissioning research and filing it away on a shelf or server, instead, we are using it to train our future decision-making systems.

I’ve been reading a lot recently on whether synthetic data is ‘good enough’ and recent research from STRAT7 run in 2025 and repeated in 2026 explored this very issue. Their work found that, while synthetic data can mirror broad response patterns, it struggles at an individual level. In particular, their research highlighted issues around logical consistency and the ability to create coherent participant-level narratives. In other words, synthetic data may be able to generate responses that sound convincing in isolation, but that does not necessarily mean those responses add up to tell a cohesive story at a participant level.

This got me wondering whether businesses have really thought about the potential risks synthetic data poses. What happens if we contaminate our systems with imperfect / incorrect synthetic data that then becomes a permanent part of an organisations knowledge bank? What happens when we feed a report with questionable data into our research library and that incorrect information has the ability to impact every future question asked of it?

Over time these answers become organisational knowledge and no one remembers where the response was first generated from. To me, this is a very real risk and one that could be incredibly costly if brand / acquisition strategies, NPD, advertising decisions are based on incorrect data. Once that data is put into the research library it’s very difficult, nigh on impossible, to remove it.

Data quality isn’t a new topic, but in a world of AI-powered insight systems it feels more important than ever that clients have confidence to trust the data we are providing. With bots and fraud at a high, we need to be able to validate participants are who they say they are, that their answers reflect lived experience and that their behaviours and attitudes come from real world contexts, because if we can’t verify the voices behind the data, how can we expect clients to trust it and make important business decisions from it?

AI has the potential to unlock decades of accumulated learning and make research more valuable than ever before, but the quality of those future insights will only ever be as good as the data that is fed in. Before we rush to populate systems with every report, dataset or transcript we can find, we should be asking if we are certain that the data we are training it on belongs to real people. If research is to increasingly live forever, we can’t be surprised to know that if we put garbage into our systems, we have to expect to get it back out, again and again and again.

Share This News