May 29th, 2013
By Bob Relihan, Senior Vice President
Mark Harrington has an interesting note under the title, “Is Surveying Obsolete?” I have a knee-jerk reaction to these articles about the inherent biases of survey research and the folly of asking consumers what they want. Of course consumers can’t always articulate what they want. Their answers can be perfectly sincere, as well as misleading, incomplete, and self-serving. It is the job of analysis, not the consumer, to illuminate their wants and needs.
Harrington continues that in an era of big data, what marketers need to do is observe their customers to assess their true desires. I will leave the question of whether consumer behavior is any more true or accurate than their pronouncements for another day. It is true that we now have an unprecedented opportunity to “observe” a large number of consumers in a variety of “locations,” engaged in multiple relationships with products and friends. And, what many marketers have always wanted to do is immerse themselves in this reality.
But, marketers have also been concerned about the cost of research, and this has resulted in the Rise of D-I-Y research and the tools that make it possible. Yesterday, someone passed on to me a nugget of information he had gleaned from a few questions he had asked using Survey Monkey. And, there are much more sophisticated tools available. But, the question is, “if surveying is obsolete, why isn’t D-I-Y research obsolete?” They have at their core the same “problem”; they both rely on consumers to report their wants and needs.
True D-I-Y research, the D-I-Y of the future, will be a set of tools and services that lets the marketer truly get up to his or her elbows in the lives of consumers.
- It will aggregate data from multiple sources.
- And, it will be able to aggregate and make connections within this data stream over time.
- It will provide tools that allow the marketer to enter the stream and test hypotheses.
- It will provide tools to analyze the stream of consumer behavior and identify trends and themes.
- It will correlate behaviors and make predictions based on these correlations. Ultimately, this kind of research will look more like Moneyball than social science research.
Old-fashioned D-I-Y research is comparatively simple and straightforward. This new D-I-Y is not. Old-fashioned D-I-Y seemed to write the traditional MR industry out of the script. This new vision of D-I-Y, which I prefer to call I-Y (Immerse Yourself), will have plenty of room for the Industry.
- Data streams will need to be aggregated and curated.
- The appropriate tools will need to be assembled into a usable arsenal that can distinguish the signal from the noise.
- The ultimate needs of the marketer need to be synthesized into the functioning of those tools. They will need to be tuned and tweaked.
- Since the immersion is on-going, there will need to be a kind of institution memory that makes sense of the insights and correlations over time.
In the future, the industry will be less involved in the realm of the craftsman, finely honing designs, data, and results and more in the area of event management. The researcher will be an impresario, a director, managing and coordinating all the pieces the marketer will need to be fully immersed in the consumer experience.
May 23rd, 2013
By Bob Relihan, Senior Vice President
The title is meant to be a bit clever. I am not talking about actual focus groups; I am referring to the term. I have always suspected that focus group was used to refer to any casual, open-ended, small sample, non-projectable research method. And, I always thought that those who used the term this way were misinformed. Well, I have to throw in the towel. I have no better authority than the Pew Research Center.
Its recent project exploring teens, their use of social media, and their attitudes to on-line privacy has generated a good deal of buzz in the press. It merits an extended discussion in the future. But, what caught my eye was the research method. There was a phone survey, twenty-four focus groups conducted in four markets, and “two online focus groups…conducted as an asynchronous threaded discussion over three days using an online platform, and the participants were asked to log in twice per day.”
This is what we at C+R Research call a short-term online community. To be sure, we conduct focus groups. We may even conduct focus groups online. But, we are careful to distinguish between the two methods — focus groups and communities. We believe each has its strengths that make each appropriate to particular situations and sets of issues.
- Focus Groups are great when consumers need to be pressed on their reactions by the give-and-take of face-to-face interaction. This is true when it is important to assess how emotionally committed they are in their reactions to something such as a new product concept.
- Online communities make it easier for consumers to describe their behavior and feelings at length, often in a nurturing, unthreatening environment. So, when it is important to explore the behavior of consumers and their emotional involvement in that behavior, online communities are just the ticket. When I want to know the needs that drive category usage and explore the gaps, an online community is a good place to start.
That is why experience is so important. When the marketing manager says, “We need to do focus groups,” he/she probably means we need to collect information in a way that isn’t a survey. It is the experienced qualitative researcher, familiar with a range of methods and categories, who can determine whether those “focus groups” should be online communities, individual interviews, ethnographies, in-store observations, or, yes, focus groups.
April 3rd, 2013
By Walt Dickie
I have argued that change in the marketing research industry is driven by changes in the corporate communication culture and in corporate decision-making protocols. The first happens frequently, and we see it every day. The second, however, occurs less frequently, but it can happen instantly and represent a sea of change. But, what will help us predict the timing or nature of this kind of change? The answer – at least the answer predicted by this model – is that change will come from developments in the greater corporate management environment, which alone has both the power and the interest to alter corporate decision-making structures, rather from change within the marketing department alone.
Let’s look at this a little more closely. What kind of information do current corporate decision-making protocols demand? This is a little tricky to answer succinctly; it seems as though it might require a recitation of all of the types and styles of MR projects that clients commission. But consider that markets as a whole are abstract entities and any analysis or recommendations about them must, out of necessity, be inferential. You can’t go out and talk to a “market” directly; you have to infer its properties. Customers – consumers and businesses – on the other hand, can be addressed directly although you may also infer much about them from various kinds of data.
Looking back at the 30-some years I’ve spent in MR, one repeated demand from clients when customers, rather than whole markets, are concerned, and, one that has only increased over the years, has been for “direct evidence” with “face validity” that bears immediately upon the decision to be made. More simply: modern corporations demand and pride themselves for demanding a certain style of evidence for “evidence-based decision making” when addressing customer-level decisions, and one of MR’s jobs has been to focus increasingly on the collection of data that plainly and, on its face, bears on the customer decision at hand.
So, following the reasoning I’ve laid out so far, we would predict that (a) any new approach that can be reasonably represented as providing more direct evidence for or against the specific customer-level decision at hand will be supported by clients, at least tentatively pending further evidence, and (b) any new approach that does not fit the model of “direct evidence” will wither on the vine unless client companies implement new decision-making protocols or new client companies appear based on different protocols.
I think this perspective helps explain the periodic rush to methods for obtaining consumer data more “directly,” “precisely,” and “scientifically” than the opinion-based methods that MR has always relied on. If, indeed, neuroscience, eye movement, facial expression, galvanic skin response, or quantum entangled ethnography were shown to be more capable of providing unmediated access to consumers’ “true,” interior experience of a product, then that approach would be understood as obviously superior in supporting marketing decisions and would, in fact, sweep the industry. This has not happened yet, however, despite repeated trials.
Will the inferential analysis of consumer behavior – the statistical modeling of customer data, browsing behavior, location, real-world shopping behavior, social media networks, and on- and offline purchasing – prove better at this? The jury is waiting and, of course, some of those “early results” for “Big Data” are tantalizing, but the larger problem seems to be that inferential statistical models are a poor fit for existing corporate decision support systems for customer-level decisions.
Senior-level corporate thinkers and decision makers express regular displeasure with their MR suppliers – and, by extension, their own internal MR departments – for failing to innovate and adapt to powerful new models for generating insight because most customer decision-making protocols favor a “direct” style of customer interrogation while MR departments continue to commission fairly traditional research designs.
What this means for the MR industry as a whole
I draw three main conclusions from this argument:
- The pace of change in MR data collection will generally match the pace of change in consumer communication technologies and norms and will be driven by new whatever opportunities and restrictions that accompany those technologies and norms. Since both are currently undergoing rapid change driven principally, but not exclusively, by mobile technology, methodological change will be both rapid and thoroughgoing. Companies that do not stay close to the leading edge of this change will be overtaken by it and suffer in the long run.
- Client pressure on MR is now and will likely continue to generally be a conservative, stabilizing force, especially for customer-level decisions. MR companies whose businesses are based on customer-level decision support will see most clients sending RFPs implicitly (if not explicitly) requiring traditional project designs using now-standard “evidence based Q&A” approaches providing “face validity.”Clients will, of course, continue to press for lower costs and shorter timelines. They may adopt “DIY” approaches quite broadly bringing evidence collection closer to the decision, and may require methods based on new consumer communication approaches as they recognize the pace of change in this area. But, radically new research approaches using new kinds of data and reliant on inferential modeling rather than direct interrogation will not occur at anything like the same pace.
- Clients that do, at some point, begin to demand that marketing decision-making incorporate and integrate a wider range of modeled consumer behavioral data may change “overnight” from supporting a traditional “evidence-based Q&A” decision model, but only if driven by top-down, wholesale corporate re-structuring of the decision-making environment. MR firms doing business with these clients may be asked to conduct research that incorporates continued efforts based on “evidence based Q&A” with “big” customer data from CRM systems, social media, purchase data, etc., or they may find themselves frozen out by new suppliers whose methods eliminate the need for traditional customer-level MR decision support research.
MR firms that do not stay close to the leading edge of “big data” integration and are not already known for their abilities and interest in this area will have little chance of negotiating this transition. But, not all companies will follow this path and those that do may do so over an extended a period of time. Still, it is quite possible that the pace of change, once established, will accelerate and become essentially universal in a handful of years as new decision models penetrate management training programs and become established in the corporate sphere.
March 29th, 2013
By Walt Dickie
Most marketing researchers are like sports fans: they’re not content to watch the game and see that Team A beat Team B last weekend. They need to work out a story about the strengths and weaknesses of the teams and develop a prediction about what that implies for next week’s games and the teams’ season performances.
I think this personality quirk explains at least some of the current obsession about the industry’s future. Some days it seems like every MR person in the universe is so busy reading and writing blog posts, making and listening to conference presentations, and sniffing out the next clue to the “Next Big Thing” or the “End of MR as We Know It” that no one is getting any paid work done. On this charge, by the way, I’m as guilty as the next guy.
But I’m conscious of feeling a growing antsiness about all of this − not because I think we’ve all gone overboard and should stop, but because I’m increasingly aware that the theory driving most of the blogging, conferencing, and sniffing is way too simplistic.
Here’s how I’d summarize the general approach behind most of the MR blogging I read (and some I’ve written):
There is a new technology/analytic approach/data gathering technique that has produced some very impressive results. A small number of innovative suppliers and their forward-thinking clients have been quietly developing this for a while now, and their efforts are beginning to move beyond the experimental stage. Real tools and platforms are starting to appear, and the leading edge tools and platforms are now coming out of beta and attracting both more investment and more client interest. The buzz is growing among clients/suppliers that this new approach could very well revolutionize/eliminate marketing research as we know it …
What is actually driving this model? What is the underlying mechanism that’s being proposed to drive the model from “some very impressive results” to “revolutionize/eliminate marketing research?”
As anyone will tell you who has ever tried to manage something from the “very impressive results” stage to an actual ongoing business, much less a business capable of overthrowing an industry, early success doesn’t come close to guaranteeing sustainability. A theory that can be reduced to neophilia – Look! A new thing! New things rule! – just isn’t adequate.
So what would a more complete theory of change in marketing research look like?
A good place to start with many questions like this one is to ask, “What defines marketing research as a distinctive activity and business?” I’ve thought about this question a lot over the years, and here’s the best answer I’ve come up with:
“MR is research conducted among the consumers/businesses that purchase or may potentially purchase a product/service, and is carried out to support a corporate marketing decision-making process.”
There are two important points in this definition:
- It locates MR as a business between a paying client on the one hand and a customer base on the other. MR plays the role of middleman communicating between these two poles when the client is engaged in making decisions about the customer market.
- It distinguishes MR from other kinds of commercial research by specifying the particular institutional role it serves: the client’s marketing function. Other kinds of commercial research, even if they involve the client’s customer base, fall under some other “research” umbrella, not MR.
The usefulness of this approach that it gives us leverage to predict where the drivers of change in MR are likely to be found: consumer communication and corporate decision making.
I’ve long argued that MR should be understood as being driven by communications – You can go so far as to say that at least the data collection side of the business can be best understood by recasting it in communications terms. How do we communicate with consumers in a way that will enable them to communicate their perceptions, opinions, beliefs, and behaviors so that we can produce, and subsequently communicate, useful analyses and guidance for our clients?
Not only the way we communicate with consumers, but also the content of those communications are determined by the mechanisms, technologies, and norms of communication in the culture. This is one of the drivers of change: we follow as consumers drop telephone and print and take up the web, email, texting/messaging, and social networking.
The same process also drives what we can communicate about: as consumers expand the sphere across which they are comfortable communicating and sharing personal information, they become more comfortable sharing those things with us. This is one way to understand some of the hesitancy we encounter from time to time.
Corporate Decision Support
In most large successful companies, the process of making marketing decisions is not free-form, intuitive, or invented on the fly. Most companies have expended a significant amount of thought and effort developing protocols for making decisions: fixed, well-defined procedures to be followed, questions to be asked, tests to be performed, information to be obtained, and criteria to be met.
Corporate marketing decisions can be seen as being made at two levels: a “higher,” more general level that concerns markets as a whole – their composition, structure, dynamics, and prospects – and a “lower,” more specific level that concerns the marketing of specific products/services within markets. “Higher level” research generally supports broad, strategic decisions; “lower level” research generally supports narrower, more tactical decisions – although this distinction is not always sharply drawn. In order to easily differentiate between the two levels, I’ll refer to these as “market level” research and “customer level” research in what follows.
These protocols are embodied in standard operating procedures, and are followed as a matter of course. Client teams negotiate exactly how to apply these protocols to the specific issues at hand, what corporate resources to expend in following them for a specific case, and how to fit that process into larger timelines. But they don’t question or re-invent the protocols themselves as a normal part of their business responsibilities.
Predicting Change in MR
Considering the different contributions of communication norms and decision protocols and the way these two key drivers evolve lead to what may be a somewhat startling conclusion:
- The aspects of marketing research driven by the cultural norms of communication – primarily the data collection side of the business – will change regularly and, possibly, rapidly in periods of rapid changes in communication. Such changes will be heavily driven by the processes of development and adoption of new technology.
- The aspects of marketing research driven by decision-making protocols – client RFPs, many aspects of study design, analysis, and reporting – will tend to change slowly because corporate decision-making standards are created by corporations to interlock with the larger corporate management environment of which marketing is but a small part.
In other words, rapidly changing new communications technology will be put to the service of unchanging corporate needs for some unspecified, but significant period of time. Put more fancifully and colorfully: even if quantum entanglement hyper-neurology is adopted as a data gathering technique, the MR industry will respond by adapting traditional project designs to use quantum entanglement hyper-neurological methods rather than explore radically different approaches made possible by QEHN as long as client decision-making protocols demand the same kind of output – QEHN will be used to evaluate large attribute grids with 10-point scales.
March 19th, 2013
By Jorge Martinez, Director, LatinoEyes
When C+R first began conducting Hispanic research, one of the difficulties was finding participants. The methods we were using within the general population were not effective. We would turn to dedicated recruiters who were wired in to the local Hispanic communities to find focus group participants. At times, they would shepherd the participants to the facility as a group in a van. It was complicated and dicey.
We have been slowly been moving away from such unusual tactics. And, it is with a good deal of excitement that we read the latest report from the Pew Research’s Hispanic Center indicating that the Hispanic population in the US is closing the “digital divide.” Hispanics’ use of the internet (78%) ids equal to that of the African American community and now approaches the level (87%) of the white population. Their use of the internet (86%) is almost equal to that of non-Hispanic whites (90%) and edges above African-Americans (84%).
What is most interesting about Hispanics as they cross the digital divide is that they are in the vanguard of the transition to mobile. A higher proportion of them (76%) access the internet with a cellphone or other handheld device than whites (60%), and even slightly more than African-Americans (73%). And for a variety of reasons, they are more likely to live in cellphone only households (47%) than either African Americans (38%) or non-Hispanic whites (30%).
So, it is clear that efforts to understand the tastes, preferences, and attitudes of today’s Hispanics can take place in the digital space and, increasingly, in the mobile leading edge of that world. But, doing so will still require keeping in mind some of the basic principles we have observed exploring the Hispanics community with more conventional means.
- Hispanic participants still need to be given language options when they participate in a survey or an online community. It is important to remember that fluency in either language does not define Hispanic identity, and that research with Hispanics does not mean that research must be done exclusively in Spanish. Self-perceptions of language ability do not always match actual competency. Some desire to use Spanish even beyond the limits of their actual competency. It may even make sense in a qualitative environment to allow participants to move back and forth between English and Spanish guided by moderators who are equally fluid.
- We still need to be attentive and accommodate contextual and cultural differences within the Latino community. Even though Spanish may be the common language the range of linguistic and cultural variation is considerable and can present even more of a problem in a communications medium that feels as casual as “mobile.”
- Our experience conducting qualitative research suggests how important relationships and the social dynamic are within the Hispanic community. When interacting with Latinos in a digital environment, we need to provide space for this interpersonal dynamic if we are going to generate meaningful insights.
Online methods may not be appropriate for all research into the Hispanic community. More traditional, face-to-face approaches are still required for reaching older, unacculturated Hispanics. Yet, we are now in an era when digital approaches can be as successful with Hispanic respondents and with any other group.
February 14th, 2013
By Shaili Bhatt, Senior Analyst
In this era of over-sharing, curated storytelling is imperative. While many of us own and use smartphones as cameras, it’s a challenge to remember to do something with the pictures and videos that we capture with these devices.
Many of us love to capture pictures and videos on our phones, and more often than not, we try to publish the most irresistible moments in our social galleries. (There’s an undeniable sense of accomplishment when I can share my stories and memories—perhaps you can relate.)
This is no different for our market research projects. If anything, it’s even more important to share key pictures and videos from our projects to help tell the story across clients’ reports and presentations.
Whether we want to make an album, an elaborate scrapbook, float it all in the Cloud on Facebook, Twitter, or Instagram, or deliver an outstanding report or presentation that really gets to the heart of the story with consumers and clients—most DIY options can be overwhelming, expensive, and for some, it can feel like a chore.
It is time to rethink the way we share mobile pictures and videos—and consume other mobile media—and look for tech-savvy time-savers.
Recently, I was excited to run across a bunch of new, free iPhone apps with robust movie-making and storytelling capabilities: Qwiki, Givit, Magisto, Viddy and Splice. These tools allow anyone with an iPhone to turn pictures and videos into a brief movie that you can share in minutes! On my way home last night, I snapped some pictures and videos of advertising paraphernalia that I used to created this video collage (click image for link).
Again, I got that research-geek thrill to uncover how these tools could potentially benefit all areas of market research:
- “On-demand” Movie/Video Collage Activity
Our participants are able to capture so many timely moments for us on a variety of mobile devices (smartphones, iPads, digital cameras, you name it!). It’s really up to us to deliver a system that organizes and focuses all of this data.
Qwiki is my favorite app of the bunch, as it automates the picture/video selection process into a one-click movie. The app works by automatically stitching together media from the iPhone camera roll and creates a 30-second to one-minute mashup from a certain day or album.
Song selection for the Qwiki occurs from the phone’s music library or from “soundtracks” preloaded in the app. Perfectionists and those of us with additional interest can easily play around with media configurations and change the audio track, which gives us an even deeper look into the mood of the visuals.
Indeed, video collages and movie mashups can bring impressive creativity and flexibility to qualitative research.
- Introductions/Warm-up: share a brief video collage or slideshow of their family, interests, typical day
- Homework or “On-demand” Movie/Video Collage Activity: create video collages and movie mash-up from experiences captured on their phones
- Reporting: integrate a video collage (that we’ve created) of participants’ key pictures and videos for a quick debrief/recap
- Presentations: show a similar video collage to get the discussion started, bringing the project to life!
The seamlessness of such videos add richness and energy to our stories in all of the ways above, as well as any others that we can dream up.
The days of creative departments and contracting with a video editor are by no means “over.” These apps lack the ability to output a professionally designed highlight reel with exact precision, multiple formats, or perfect image resolution, and its related audio effects and capabilities are limited, at best.
Still, these new apps are remarkably efficient, and we’ve certainly found another cool solution to elevate our perceptions of the smarts in our smartphones! Telling a great story is even easier as we ramp up our use of video collages—and make them faster, better and cheaper with today’s mobile capabilities.
What are your suggestions and feedback around these new apps? How are you integrating videos into your current market research efforts? Post a comment to share what you think!
January 9th, 2013
By Bob Relihan, Senior Vice President
Walt Dickie had done a very nice job of knitting together the trends in the adoption of various electronic devices. Certainly PCs are flattening out and will eventually decline. And, I agree there will be a time when virtually every cell phone is a SmartPhone. Walt also plots a curve that predicts exponential growth in the tablet/e-reader market, but backs off from the implications. “I’ve gone with a growth curve that can’t be right in the long term – it has to flatten out – but might be okay in the short term.”
I am not so sure.
Now, it is likely true that all but a few high flyers and the tech obsessed (as well as those involved in illicit activities) will ever have more than one smart phone. The device, after all, is tied to one’s personal phone number. But, the same constraining logic does not apply to tablets and e-readers, particularly when they merge into one category with vaguely similar features and price points ranging from $79 to over $800.
So, in the next few years, as more and more users acquire new tablets with better features and still have serviceable old ones on hand, it is easy to imagine a home with a first generation Kindle by the beside, an older iPad on the kitchen table for reading the news and checking the weather in the morning, and the latest tablet sitting on the coffee table in front of the television. Will there be a television? Why carry a tablet with you? You can have one wherever you turn.
There was a time when the household was dominated by one large console television in the living room. Over the years conducting focus groups, I have asked in passing, “So, how many TVs do you have?” Seven is no longer an uncommon answer…in a household of two. A future with a tablet in every room is not that far fetched.
By Walt Dickie, Executive Vice President
I love the Pew Research Center, especially their Internet & American Life Project. I can always find something interesting to think about on their website, and I admire their invariably solid methods. We use Pew data to make strategic decisions, but we also go to Pew for inspiration when imagining future scenarios.
A recent Pew post, including data through September 2012, based on a long-running tracking study and a more recent update on smartphone ownership brings together information about consumer ownership of desktop computers, laptops, cell phones, smartphones, and tablets among U.S. adults.
Some of the most important parts of the dataset are still a bit sketchy. Not because Pew didn’t do their usual excellent job collecting it, but because the number of data points is still pretty limited. In the spirit of fooling around with numbers and the informality of blogging, I decided to analyze this data to generate some hypotheses about smartphone and tablet adoption. The analysis that follows plays somewhat fast and loose–extrapolating trends beyond the range of the data and basing these trends on an inadequate number of data points. This sort of thing is fun and may be stimulating; it is not conclusive, nor is it meant to be.
Here’s a selection from Pew’s device ownership data plotted together on a single graph:
Desktops + Laptops (“PCs”)
I aggregated Pew’s data on desktop and laptop ownership to create this curve for traditional “personal computers,” which goes above 100% because it’s quite possible for someone to own one or more of each species. If you look at the original Pew data for laptops and desktops separately (not shown here), you’ll see that laptop sales are still rising while desktop sales are dropping precipitously. The curve shown represents a leveling-off of PC ownership at somewhere between 1.1 and 1.2 PC’s per household, which the Pew data suggests will be mostly laptops as desktops die and are not replaced.
I feel reasonably okay with fitting this flattening curve to the cellphone data, and although projecting curves forward in time like this always gives me the willies, the result looks at least somewhat plausible. Cell phone ownership is clearly flattening out with something like 15% of the population being reported as doing without. Cell ownership will probably never hit 100%–landline phones never did–but it will certainly get further into the 80s, and maybe even into the 90s, as landlines pretty much fade away and all the people who grew up in the pre-cell phone era disappear. Of all the curves on this graph, I think this one may be the most realistic.
Fitting a curve to the four data points on smartphone ownership is clearly beyond the pale, but having decided to work with what we’ve got why not do it anyway? An exponential growth curve may capture what, by all accounts, has been a startling adoption rate, but of course no trend, even a really powerful one, will continue into the future without slowing-probably more noticeably than is predicted here. Smartphone ownership seems to have paused in the middle of this year, but with the phenomenal sales figures being reported for the iPhone 5 maybe predicting penetration to continue its strong growth isn’t so bad, at least in the short term. In any case, I wanted an aggressive scenario to explore the impact of smart phone growth, and that’s what this curve represents.
Not many data points (5) here either but tablet adoption sure sounds like it’s accelerating according to the news reports, and an exponential curve fits the existing data almost perfectly. I’m writing this immediately in the wake of “Cyber Monday,” and the news outlets and blogosphere are reporting tales of tablet frenzy following the debut of the iPad mini and Amazon Fire. Again, whether exponential growth will be sustained is questionable, but it doesn’t seem wildly off to estimate that we’ll experience that kind of growth in the short term.
Interestingly, the curve of e-Reader ownership shows an almost identical rate and pattern of growth, which raises the question of whether these are one species or two. Although tablets and readers started out as separate species, it’s hard to imagine how they continue to evolve without merging into a functional/price continuum competing in a single market, even if some of them continue to be distinguished by very different screen technologies adapting them for use under different lighting levels. If the two combine into a single product line the case for exponential growth may be strengthened.
So, again, with some basis in statistics and observation, but mostly because I want to create a best-case scenario for new device adoption, I’ve gone with a growth curve that can’t be right in the long term – it has to flatten out – but might be okay in the short term.
Using some obviously bogus methods but maintaining at least a nodding relationship to “reality,” I hereby predict that smartphones will essentially eliminate “feature phones” from the cell phone marketplace in about 3 years; at which point, everyone who owns a cell phone—something on the order of 85% of US adults—may own a smartphone. I further predict that in about 1 year ownership of tablets in the US will equal ownership of traditional PCs, with most households owning one or more mini/maxi/reader “tablets.”
All of which means that MR has a very short window for adapting its data collection methods from a PC-centric paradigm to one centered on smartphones and tablets.
This will be one of the biggest challenges in our immediate future for both C+R and in the MR industry as a whole. We simply have to get “mobile” right. As long as our clients demand data-driven insights, we’re going to depend on consumers being willing to share their perceptions and opinions with us, and we depend on technological means to collect that data.
Looking back, the conversion from phone/mall/mail survey data collection to online methods at the end of the 90s seemed like a major revolution that upended almost everything and rang in a new era. But, in hindsight, although the mechanisms of research changed a lot during that period, what now seems to stand out is that the basic paradigm of question-and-answer surveys changed very little. Other than porting surveys from “CRT terminals” in the phone room to PC screens in the nation’s family rooms, dens, bedrooms, and kitchens, the underlying form of the survey hardly changed at all. Surveys grew images, videos, and Flash widgets but the great majority of MR surveys created today for online administration could be ported (back) to the phone room quickly and with ease.
But mobile is going to be different. Cellphone use is dominated by short interactions while few online surveys take less than 20 minutes (and many take more). Today’s surveys, though they can be taken using a mobile device, are a miserable experience because the industry mindset is still fixated on the PC. (My friend and colleague, Bob Relihan, says that the most miserable experience on a cell phone is trying to fill out a web form, and online surveys are composed of one web form after another.) A whole new survey paradigm will have to be invented to model a virtual “long survey” from a series of very short interactions on a mobile device or something resembling a Google Survey. And the sampling industry is going to have to reinvent itself once again.
The “PC revolution” (the 80s) and the “Web revolution” (the 90s) are going to give way to the “Mobile revolution.” The question is “When?” and the answer could easily be, “Very soon!”
December 19th, 2012
By Walt Dickie, Senior Vice President
(If you haven’t already, be sure to read part 1 to this blog post.)
I’m assuming that if you’re reading this you’re probably a marketing researcher. If that’s the case, you probably have access to a bunch of survey questionnaire. Look on your hard drive or reach into your file drawer if you’re old school and take out three or four. Look them over and back up far enough so you’re looking at their general structure rather than the details. What do you see? I’ll bet they all share a similar approach:
- First, you almost certainly see a bunch of screening questions designed to eliminate all but a pre-defined target group from taking the main survey. This makes sense because you pay for sample, pay incentives, and pay for resources to administer, manage, and analyze the survey and you don’t want to waste resources.
- Next your surveys probably have a section or two devoted to category-level questions, so you can profile results based on pre-established dimensions of opinions about and usage within whatever product/service category the survey focuses on. Again, this makes sense. It establishes context and in many cases may allow you to compare your findings to previous work.
- Then your surveys probably show or reveal something in order to get respondents’ reactions to it. Purchase interest. Likes and dislikes. Batteries of questions dissecting emotional reactions. Maybe some open-ended questions to lay bare the underpinnings of those emotions. Maybe you’ll go around a couple of times on the reveal/react carousel. Or dissect the combinatorial possibilities with some conjoint or discrete choice questions. All sensible choices. This is the meat of the survey and you want to be able to mine it for key findings and “insights,” which are, after all, what the client is paying for.
- Finally, you’ll almost surely find a bunch of classification questions – again, quite a sensible thing to include for establishing context and tying your work back to the client’s tracking data and historical survey archive.
Everything in your surveys is sensible, serious, and workmanlike.
But back up and consider your surveys from the perspective of someone who, somehow, had never seen anything like them before. There’s really quite a lot about your respondents and what they do and think that your surveys make to mention of. Your surveys contain such a limited set of choices!
There is, for instance, no information at all about what their friends and family members think, or what they would say about either your product or new idea. Nothing about what your respondents search for on Google or read online. Nothing about where they go everyday as they travel around the places they live, or what they see or what they do there. Nothing about their inner lives: what they love, long for, dream about, pledge allegiance to, dislike, despise, or denigrate. Nothing about their pasts, really, or about their futures. And there’s nothing about them that they are not or cannot become conscious of and respond to in answer to your question. Not to mention the problem that all of your questions come in prefab form, so the issues they address are constrained to the issues the client already recognizes. And let’s not forget that there’s no information from anyone who fell outside of your carefully limited screening quotas. How can you be sure that there are no discoveries to be made among them?
You’ll object that you didn’t omit any of that because you aren’t interested in experimenting with it or would reject the possibility of finding value out of hand, but you have a limited budget and only so much time to spend. The client won’t pay for experimentation and, besides, there’s a decision to be made. The client has a lot riding on the research and they’ve given a good deal of thought to the question of what they need to know to make that decision. You need and want to give them good value for their money.
I’d like to suggest that this is a powerful example of what drives bubbles and echo chambers. Marketing research has evolved to serve structured corporate decision making and that process has evolved to demand the inputs that marketing research provides.
Corporate decision making and marketing research have coevolved the MR survey with a closed set of constructs and almost ritualistic format. Unfortunately, the evolution of surveys has been driven by the imperatives of the Red Queen – running faster and faster to stay in the same place, becoming an ever-closer fit with the needs of corporate decision making, budgets, and timelines. Each step has not been determined by the possibilities of utilizing new data sources, analytic approaches, or even accurately predicting future behavior, but by the ever-closer interlocking of the decision process and the information machine that feeds it.
Like Republican pundits talking to Republican audiences and eventually creating a closed worldview that, among other things, mis-called the election, we’ve created a mutually evolved worldview that has satisfied us and our clients for many years, but, as the CEOs keep telling us, often seems to fail to predict the real world and doesn’t seem to be getting any better.
Will the appearance of data nerds bearing data on things like clickstreams, search histories, location, social influence, sentiment analysis, and facial expressions – complex real-word behavior and data that isn’t based on the ability to frame a conscious answer to a prefabricated question – be the beginning of the opening-up of that closed co-evolved world? Will MR break off the echoing conversation before our clients do? Will it take the equivalent of a public defeat on a national election stage to turn things around, or will MR put the genuine technical expertise, hard work, and ingenuity to work before we have to cobble together a last minute concession speech?
December 17th, 2012
By Walt Dickie, Executive Vice President
I’ve been working in the MR industry since 1978, and the one unchanging theme of that long period has been the constant complaint by the senior corporate executives – who fund the industry by their demand for research-based decision making – that MR just isn’t very good at identifying opportunities or pointing to the new products, services, and ventures with the best chances of successfully capitalizing on them.
I remember being surprised when I first encountered a CEO denouncing the entire MR enterprise at a major conference – which probably happened sometime within my first or second year. Was my new, desperately needed first job after grad school, and maybe the whole industry, going to go down in flames before I even got started? It hasn’t but the drumbeat of C-suite dissatisfaction has never lessened.
This all came to me as I was reading one of the many articles from the “nerdiest election” in which Romney had not prepared a concession speech because – by all the accounts I’ve seen – he didn’t think he’d need one. It sounds like Romney really got blindsided. Not only did he not have a concession speech held in reserve, but he also planned to celebrate the election with a fireworks display over Boston Harbor and his campaign even had a “Romney Wins” website up and ready. (Someone posted screen grabs, of course.)
As Slate’s John Dickerson said, “He got the numbers wrong … in the end Romney and Ryan had to watch CNN to find out how their campaign was doing.”
The blog posts and news stories lay blame on what David Frum labeled the ”conservative entertainment complex.” And it looks like the Romney’s “marketing research” group followed the media’s lead in asking the questions they wanted asked, hearing the answers they wanted to hear, and reinforcing an internal viewpoint that, in the end, failed as MR and left the CEO “gobsmacked” and angry.
I read all of this with the shock of recognition. My mind snapped back to 1985 and the introduction of New Coke. All the research that had been done! And, the magnitude of the disaster that followed! The astonishment of everyone inside Coca-Cola that their carefully constructed edifice had been built on sand. I had seen all of it and had enjoyed having the inside scoop thanks to two colleagues who had come from Coke’s MR department and understood the backstory, which had included a huge research effort.
The post-election stories seem to be revealing a distinctive, internal worldview shared by the campaign and its media supporters. Republican pollster, Whit Ayres, described the research being drive by “rosy assumptions on a likely electorate…at…substantial variance with recent history.”
An “echo chamber” bubble developed when the campaign research staff and its clients elaborated a shared narrative about polling and sampling methods and how to interpret results. Outside that bubble, the academic social scientists and media stat nerds – with Nate Silver as their symbolic leader – were using different methods, asking different questions, and interpreting their findings differently.
They were right in the end, and the people in the bubble were wrong.
A case can be made that MR and our clients have created a similar bubble, in which we talk to each other in an echo chamber. And, a case can be made that the academic social scientists and techie stat nerds are “threatening” traditional MR with everything – including Big Data, social media and web based behavioral analytics, location data, remote facial analysis, and eye tracking – living in an entirely different world.
The GRIT survey as well as reports coming out of consultancies like Cambiar (site registration required), say clearly that corporate research departments are eagerly welcoming all the new “non-MR” vendors knocking on their doors. What’s worrisome is what would happen if the CEOs start sending more work to the guys outside the MR world, and they started having some successes calling the game. That’s what the voices that expect the leading “MR” vendors of the end of the decade to be companies like Google, Facebook, and Twitter.
Why didn’t all of these new approaches arise out of marketing research? At the very least, why weren’t traditional marketing research companies their earliest and most eager adopters? Why didn’t clients hear about all of these developments from their MR vendors? Is it because the conversation was “closed” and things like that simply had no place?
I’m not arguing that MR is operating in bad faith – only that they and their client audiences may have constructed a particular shared view of commissioned and conducted research that has become closed, limited, and overly rigid.
Marketing researchers are almost universally serious, sober people who see themselves as technical experts in a field that demands hard work, clear thinking, and ingenuity to provide vital information to decision makers while often suffering little respect, diminished budgets, and constricting timelines. But make no mistake about it, marketing research has been working with the same basic approaches for at least a couple of generations. I made a stab at some of the qualitative issues in a previous post, so in part 2 to this post, I will examine a few things about quantitative research.