Research is not reality

Do you believe that qualitative research reflects ‘reality’?  I don’t, and I can’t say it bothers me.

Isn’t this an irresponsible position for a qualitative researcher to adopt?  How does it help an industry that is striving to achieve the credibility that can get it “into the boardroom” and have John Humphrys take it seriously over breakfast?  Surely it’s madness to be saying “reality doesn’t matter”.

First of all, let me define my terms.  I’m not referring to exploratory studies, where going into the ‘real world’ environment can be helpful or even essential.  I am thinking about development research, where I find the call to make qualitative research “closer to reality” is a common one.

There seem to be two main ways in which people want to see more “reality” in development research.  They either want to have stimulus material more highly finished so that it “better represents what the finished XYZ would be like in reality”, or to place the material “in a realistic context”, so that the consumer can encounter and appraise it in a way that is “closer to reality”.

In most cases, this desire is misguided.  Let’s take the use of highly finished stimulus material.  Finished executions always move on considerably from the creative ideas put into research, meaning that stimulus material can never represent reality, unless it is the finished product.  Furthermore, researching more finished material runs the risk of giving erroneous results.  The more ‘finished’ the stimulus looks, the more respondents take it as finished material, reacting to the executional elements rather than the idea.  Key frames for TV ideas are usually pretty poor and animatics worse, the introduction of ‘movement’ furthering the misleading impression of ‘reality’.  This is why, in cases where it is appropriate to the idea, I avoid using key frames for TV advertising development and instead uses scripts or narratives. Contrary to the orthodox view, my experience is that most people are adept at imagining scenarios from vividly written descriptions, and this helps creative ideas ‘live’ in research.

The best approach is to accept that stimulus material cannot represent reality and concentrate on trying to represent the idea clearly, rather than the ‘reality’ of its finished expression.  After all, in development research, we are not exploring the finished article, we are researching the idea.

What about the desire to put stimulus material “in a real context”?  I am sure all researchers can give innumerable examples or this, but take the suggestion that pack concepts be researched in mocked-up displays.  Why do this?  Is it to assess stand out?  Quite apart from the fact that you can’t assess impact qualitatively, such an approach takes no account of how people browse the fixture in situ, which can have a major impact on response (‘browse’ is a misnomer, given the way people pull goods from supermarket shelves on ‘autopilot’).  And, importantly, the pack will never be displayed in this way in the real world anyway, so why try to pretend it will?

In the past, I have been asked to research beer advertising concepts in a pub because it will “get respondents in the right frame of mind”.  Perhaps this would be fair enough if they only saw TV or print advertising when in the pub, but since when has this been the case?

Not only is the pursuit of ‘reality’ in development research a largely a pointless endeavour, it can be positively harmful.  Where it is done, it leads to an erroneous sense that the results are a better reflection of reality, which encourages people to treat them as more reliable and definitive.

What is driving this call for more ‘reality’ in development research? I believe it stems from a desire to make qualitative research more predictive.  Let’s be clear about this: qualitative research is fundamentally non-predictive, and any attempt to construct it in a way that suggests it might be needs to be resisted.

So, if reality doesn’t matter, what on earth are we doing in qualitative development research? In my view, qualitative development research exists to help us think.  The purpose is to experience the people who make choices within the relevant market processing brand and communication ideas within their own frames of reference.

This touches on the point where I think ‘reality’ is vital, and that’s in recruitment: our respondents must genuinely be the right people, because the whole point is to get the perspective of people who live in the market rather than speculate about it. What people tell us (verbally, through body language or through enabling techniques) helps us think about how the ideas might be working and to develop hypotheses that we can explore further.  In this way, it helps us work out how the ideas probably function.  The brand team can see things they hadn’t before, recognise new opportunities, reappraise their instincts or understand why they felt something intuitively, thus enabling them to work with it more effectively.  The team can then enhance ideas, take them in new directions, retrace their steps to find new departure points and, most importantly, find ways to make ideas better.

Of course, this thinking has to be applied in the real world.  But the understanding of the real world will typically be derived from other sources.  Research is one source, but this should be research specifically designed to understand consumer behaviour “in reality”, for example: factors influencing brand choice at POP, or ways in which TV is used in home.  Clients should be using this knowledge, plus their own experience of how consumers operate in their markets and their broader business understanding, as a context within which to think about what’s coming out of the development research.  Our research can provide clues as to the possible answer, which can only be arrived at by consideration of many other pieces of information – the research is only one piece of a jigsaw, not the whole picture.

So, if you are involved in qualitative research that aims to help develop new ideas for brands and communication, I would urge you to challenge the call for something “closer to reality”.  Point out the fallacies and dangers of attempting to build more ‘reality’ into the process.  Defend the intellectual integrity of qualitative research and the commercial value of ideas and thought, which are increasingly important as drivers of competitive advantage in the modern economy.  And accept the fact that research is only one part of the jigsaw that the brand team has to assemble.

Some end users may not like to hear it, but such an approach, well argued, stands to enhance your value and respect.