Archive for the ‘quantitative market research’ Category

By Walt Dickie, Executive Vice President

I’m not a news junkie, but I’ve always read newspapers. Having grown up and gone to college in New England, I’ve been reading the Times since I was a kid, and I can’t remember a time when www.nytimes.com wasn’t the home page on my computer.

My opinion of the Grey Lady has varied over the years. I’ve always admired how the Times tried to be the newspaper of record, even when it fell short of its own standards. And although I’ve thought they made some remarkably dumb moves over the years (see “Paywall, failure of”) I’ve increasingly come to admire the Times as an online innovator.

Although it’s commonplace today for online companies to have an API, the Times was the first major newspaper I was aware of to do so. It’s also the only newspaper I know of that has an experimental development arm, beta620. And although it’s almost impossible to imagine a news site without graphics, the Times has been especially thoughtful about the graphics they build and how they relate to the principles of journalism.

But the Times has also been working on data collection projects – I first noticed these around the last Presidential election cycle, but there may be earlier examples. I remember sending friends of mine links to an online “mood tracker” that aggregated answers to the question, “What are you concerned about right now?” into a timeline and used type size to indicate the number of people sharing a common concern. It was simple, informative, constantly shifting, and sometimes surprising.

More recently, the Times took a step further and created what looks to my eyes – practiced as they are at looking at surveys – as a truly re-imagined survey. It was in  (a) timeline.” Here’s a small screen shot of what the display looks like:a year-end piece about the future of computing, and it asked readers “to make predictions and collaboratively edit

If you hover over any of the entries, it will open up and show a longer description of the topic. Here’s what you see under “Routine Voice Interaction”:

This is a survey question – something on the order of, “In what year do you expect routine voice interaction to be available on all computing devices?” But, besides displaying the question, the Times survey shows who suggested it (they also link to information about the proposer), the current consensus answer, the number of respondents who have previously disagreed with the then-current answer, and an invitation to vote on moving the date forward or backward. Imagine how grueling this would be set up in a typical online survey. The Times version has 53 separate events – I can hardly imagine a way to make responding to a survey grid that size bearable.

A lot of things are going on here; one being better graphic design than most survey tools can support, of course. But the more I think about this, the more it strikes me that the big change is that the Times has shifted the concept of this “survey” from a model of “we ask questions, and you answer them,” to one that puts more power in the hands (and mouse) of the “respondent,” who is invited to explore this space of topics, to think about topics in a context of their own choice rather than in the sequence that the questionnaire author chose, and to ignore topics in which they have no interest and/or knowledge. The “big idea” here is a significant shift of control from the researcher to the respondent.

I encourage you to go to the site and try it – especially if you’re a tech person who knows and cares about computing. You’ll find that you don’t just respond to one topic at a time – you can, for instance, look across the items that are now all clustered around the same year and think about whether, in your mind, they should all belong together. You may advance one of the items a few years, which may make you re-think one of your earlier answers, so you go back and adjust that one to bring it into line with how you’re thinking now. And you’ll almost certainly give some items a pass.

Let me be clear here: I know the analytic reasons for questionnaire order and error messages that say, “You must answer all of the questions on this page before continuing.” And I’m not suggesting that this exercise represents the future of all surveys or that it’s even the best possible expression of a more user-driven alternative.

But ask yourself whether the ability to conduct certain kinds of statistical analysis based on certain kinds of sampling models, with control over order effects and incomplete data records always and in every case would outweigh the value of considered responses from involved respondents. The MR industry’s answer to data quality issues almost always involves a critique of survey design, so why do we continue to produce surveys that perpetuate the ones we designed for phone interviews in 1992?

By Lynne Bartos, Vice President

There is nothing more embarrassing for a researcher than to hear a client say “…this doesn’t really address the business questions that we set out to answer.” This is more common an occurrence in research reporting than most of us would care to admit. But unfortunately, much report writing these days falls short of expectations for those on the client side.  This is likely due to more emphasis on methodology or analytic technique at the expense of clear graphics, creative story-telling and actionable direction.

What often happens during the report-writing process is that market researchers have their direct research client in mind.  They neglect the fact that their direct contact must present these findings to the ultimate stakeholder in the process — someone in senior management or the head of marketing who does not function in the research realm.

We need to take conscious steps to break out of our little bubble to avoid some of the lingo that is prevalent in research circles. You know what I mean if you’ve ever found yourself presenting your findings to marketing folks.  While peppering them with terms such as “mean,” “monadic,” “DK/NS,” “latent class,” and the like, you suddenly notice the deer-in-the-headlights reaction.  Worse yet, your audience’s eyes glaze over completely.  These terms are foreign to many marketers and, frankly, most of them couldn’t care less about such things.  They simply want a viable solution to the particular business need they set out to address.

So, when writing a research report for my clients it helps me to keep a few things in mind….

Speak to Marketers in Their Language

Focus on what marketers care most about — getting customers, keeping customers, and increasing their share of the customer’s wallet.  So tell them what is meaningful to them….

• How to position their brand
• How best to price it
• Who their best prospects are and how to reach them
• What message should they be communicating
• Who are their most loyal and valuable customers
• How do they keep them loyal to their service or brand

Net, net — put some Marketing-Speak into your report, and leave out the Research-speak.

Tell a good story

A good report tells a good story.  So, how do you tell a compelling story?  Start by getting organized!

• Develop an analytic plan that focuses on business issues and objectives — the questions that need to be answered.
• Outline how the questions will be.
• Once the data is in, all team members should know how the data relates to those question, and they can craft the best story together.

Remember, every page in the report should contribute to the story!  If something doesn’t contour well with your story, stick it in the Appendix.  How many hundred-page reports have you been subjected to where the charts are all in the same order as the questionnaire?  Where is the story?

It’s also important to stick closely to your analytic plan when crafting your story. The analytic plan is what helps to keep everyone focused on why the research was conducted in the first place.

Insightful Headlines and Bullets

What I also find helpful in getting my arms around the story is to write effective bullets and headlines for the data presented.  Too many people think an insight is reiterating the numbers that are in the charts.  Remember, anyone can read the numbers on a chart – our job, as researchers, is to pull the deeper insights from seemingly obvious data.

Be Creative and Have a Llittle Fun

Creativity also comes into play!  Package the story in a creative way. No one wants to see rows and rows of data. Make the report visually appealing so you don’t intimidate those who are going to be using the findings to help drive strategy.  Avoid too much text and too many numbers.

And, don’t be afraid to insert some humor here and there. It reminds your clients that you are human and helps to lighten the tone and keep things relaxed.

Get to the Heart of It

And finally, probably the hardest part of the report process for any researcher is to get straight to the heart of it… what is the story – conclusions, implications, and recommendations.  Go to the next step to tell them what the data MEANS, and what they might consider doing to maximize their investment.

And there is nothing sweeter to a market researcher’s ears than to hear a client voice saying, “Thanks, this really addresses the business questions that we set out to answer!”