Two years ago I presented the MindTime segmentation technique (now called SmartSlices) to the CEO of a market research company and his senior research staff. Later that day he told me that, given what I had just shown them—a way of measuring and segmenting audiences by people’s most fundamental and predictive behavioral drivers, he was unsure that the panels they recruited from their database of research participants properly represented their clients’ target audiences. In fact, he realized, they truly had no idea if even their database was composed of a proper balance of the possible basic types of thinkers. Could it be that people who signed up to be on these kinds of research panels were doing so because of a shared behavioral driver, and others, who did not share their hidden need, were not represented at all?

Six months ago a leading research company who provides research communities to major brands, used our audience segmentation and intelligence technology to map one of its many market research communities. Having mapped all of one community’s members they then correlated this segmented behavioral and attitudinal data against brand sentiment data on ten top brands collected from these same community members.

The first of many insights we quickly gained was that the community they had just mapped was not at all representative of a general audience (and its behavioral make up). In fact, it was decidedly skewed towards just two of the three fundamental thinking perspectives, and predictably, these were the types of people who would willingly sign up to participate on a regular basis in a market research community. The problem was that these might not be the kinds of people whose opinions and sentiments the brand clients were interested in learning, they certainly were not a good cross-section of what you would find in an everyday population, the innovators and idea people were missing.

A few days later I was speaking to the person who had originally made the introduction for us to the branded community provider. When I told him about the growing evidence we were accumulating that these panel communities being used in market research might not be representative of an actual audience at all, and were biased towards certain kinds of thinking behaviors, he told me that to tell the “client” this would be like telling someone that their child is ugly. Fair point.

Right now, I am sitting looking at a map of 400+ respondents who were part of a panel recruited to participate in a market research survey for a major international Telcom brand. Embedded in this survey were our nine survey items—the MindTime Profile. These survey items allow us to map people’s perceptions and thinking within the MindTime framework, a powerful constant that can be used, among many other things, to develop personas in market research. The company actually conducting the survey had been instructed to recruit the research panel from small business decision makers, the brand’s target audience for a new product.

There is little doubt in my mind (and in that of our chief scientists), that once again we are seeing a decided bias in the make-up of the panel audience. It is skewed away from the very people whose reactions to the new product the brand is trying to understand (early adopters) and towards people who might predictably agree to participate in panel based audience research.

Let’s talk about that ugly baby for a moment. A mother and father cannot do anything about their baby’s unfortunate circumstance (other than becoming less shallow themselves). However, market researchers who recruit panels to conduct audience research can do something about their ‘ugly baby.’ Given that they have been aware of these biases in audience composition for a long time, although in their defense the solutions available to remedy them were inadequate for sure. Today there are much more powerful tools available to help them rethink their recruitment practices and screen to ensure these biases are accounted for. In fact, intellectual honesty now demands that this issue no longer be overlooked.

We have now identified a major methodology flaw in market research studies that do not account for the thinking style of their research panels and target audience. The results will therefore be flawed. Screening for thinking style profile data should be standard for any research and audience identification initiatives, similar to other demographics (i.e. sex, age, income, etc.). The benefit of capturing this thinking style data is that it also provides a powerful way to segment audiences by a behavioral constant.

Key Takeaways

• Thinking style identification should be performed at the panel recruitment level 
• Assure thinking style composition of research panels is comparable to target audience 
• Attitudinal and behavioral data should be correlated with thinking style to obtain predictive insights
• Collecting thinking style data should be as common practice as any other key demographic