Big Data in Market Research


By Bob Relihan, Senior Vice President

Big Data raises a number of interesting issues for the researcher. One of them is the position or value of theories and hypotheses. Marketers increasingly wish to immerse themselves in the experiences of their consumers. With Big Data and predictive analytics, we have the prospect of being able to identify associations within the behaviors of all our customers and potential customers. And, those connections will drive how we market to and communicate with them.

There is no need for a theory to explain why consumers do or want certain things. We will simply know them, directly. In the past, we relied on theories or hypotheses to give us a visceral confidence in the conclusions we drew from mere samples of consumers. The "whys" of consumer behavior helped us create the stories that knitted the data into a compelling narrative.

Evgney Morozev has an extremely useful reflection on this is in a recent article in Slate. The Boston bombing and the recent NSA revelations bring the issue of Big Data's utility to the fore. A good number of security analysts believe that data mining and predictive analytics can help identify and thwart terrorist threats. Who could be against this? If we can identify and stop a threat, what is the value of knowing why it occurred?

But, Morozev argues that the utility of data mining is less clear in other less immediate and charged areas of public policy. "Big Data ... can help us avoid occasional jolts and disturbances and, perhaps, even stop the bad guys. But it can also blind us to the fact that the problem at hand requires a more radical approach. Big Data buys us time, but it also gives us a false illusion of mastery."

He continues to make an extremely useful distinction. "We can draw a distinction here between Big Data--the stuff of numbers that thrives on correlations--and Big Narrative--a story-driven, anthropological approach that seeks to explain why things are the way they are. Big Data is cheap where Big Narrative is expensive. Big Data is clean where Big Narrative is messy. Big Data is actionable where Big Narrative is paralyzing.

"The promise of Big Data is that it allows us to avoid the pitfalls of Big Narrative. But this is also its greatest cost. With an extremely emotional issue such as terrorism, it's easy to believe that Big Data can do wonders. But once we move to more pedestrian issues, it becomes obvious that the supertool it's made out to be is a rather feeble instrument that tackles problems quite unimaginatively and unambitiously. Worse, it prevents us from having many important public debates."

From the marketing perspective, the implications are clear. Big Data and predictive analytics are tactical, not strategic, tools. They will help us respond to the here-and-now. But, they will be less successful at pointing us in the directions of new markets and new products based on evolving consumer tastes and behaviors. They certainly will not help us if there is a rupture in the continuity of the consumer experience. Looking to the future and creating new markets will require insight into why consumers do and think what they do. What social structures are evolving that will drive a new set of consumer wants and needs?