Why Custom Research?
“To get the right answers, you have to ask the right questions of the right people.”
One afternoon, Willow Research took an hour, sat down on our office couch, and discussed what custom research means to us, how we have seen it used, and what it can do for our clients.
No jargon, just conversation and good stories.
Here we talk about the value and process of doing custom research.
So, what is the process for beginning a customized research project? Let’s say my company hires you, where do we start?
Sara Parikh: Honestly, before we’re even officially hired, we’ve started the conversation to have a really comprehensive understanding of the business issues, not just the research objectives, but the business issues. And to help [clients] hone in on the information objectives.
And then we’ll make some recommendations based on those conversations. Put together a proposal, and discuss the methods we would recommend, how many interviews, what types of interviews, whether we do qualitative or quantitative or a mix.
Oftentimes there’s a back and forth where we’re fine-tuning the project. And once we’re ready to launch, there’s usually a kick-off to make sure we have what we need to execute the study. A call, or a meeting with the client. Walk through the timeline, make sure everyone knows their role, those kinds of things.
Are there any projects where the goal changed during initial discussions? The client started with one in mind, and your experience suggested a refinement of the goal or different one altogether?
SP: I think the business issue will stay the same, but the information objectives will be refined or expanded, because we have enough experience in a certain area that we can say, “This typically relates to this objective, so we want to make sure we explore that while we’re doing this.”
Vandana Razdan: We might make a recommendation. Let’s say they say, “We really want to talk to this consumer segment.” We might come back with, “You also want to include this other segment, because they might have interesting insights or they might be related in some way.”
SP: Or they might be an influencer, a big intermediary. If you’re talking to business owners about insurance, for example, maybe you also want to talk to brokers, because they’re very influential in selecting insurance providers. So, there are things like that we can talk about, in terms of the universe or the questions we’re asking. And that often happens in the beginning stages of a project.
As you’re starting to get data back, do you ever recommend a change to the project after looking at initial responses? Or additional questions, refocusing, etc.?
SP: Usually it’s not a wholesale scrapping of the project. It’s usually a refinement, or you take a deeper dive in an area. Because you’ve got a hypothesis and something comes up that seems to be surprising.
Betsy Kniffen: Or you realize respondents aren’t understanding how the question was asked and you reword it slightly.
SP: It happens all the time, particularly in qualitative work. Those discussion guides—when we’re doing qualitative interviews—are living, breathing things. And they change, really up until the last interview. Because we’re learning as we go along, and so we’re adjusting as we go along.
VR: It happens in quantitative surveys, too, occasionally. We may tweak a word here and there.
How do you decide which research method to use for a particular project?
SP: The first thing is based on the objectives, and that drives it. The information objectives. If they want to be able to project to a larger audience and segment the data, it’s typically quantitative.
VR: Or if they want a benchmark or a baseline study that they want to repeat down the road for tracking purposes.
SP: If they want a deeper understanding, if they’re more in an exploratory mode, it’s qualitative. Oftentimes, we’ll do both, where we start with qualitative to help us get a deeper understanding, and then that will help shape the survey for the quant stage, to make sure we haven’t missed anything really important. It also provides a deeper level of insight and some nice verbatim comments for the report.
VR: Quantitative research helps to validate any working hypotheses we have going in, that have evolved or emerged from the qualitative research.
SP: To take it a step further, within qualitative and quantitative, there are also multiple methods. So how do we decide that? Do we do focus groups, do we do one-on-one interviews? Do we do it by phone, do we do it in person?
If we’re talking about qualitative work, you might do a group if the group dynamic is really important. For example, if you’re looking at joint decision-making, or if you really want to get a creative process going and you want the group to generate ideas for you, that’s a really powerful tool.
But if it’s a very sensitive topic, for example, or if it’s business decision-makers and they’re not willing to share business secrets with competitors, you might do a series of one-on-one interviews. That will vary based on the target market and the objectives.
Phone or in-person is partly dependent on who you’re trying to reach, whether you need to show them something, whether the client wants a backroom experience, those kinds of things. And quantitative, it’s phone, online, mall, mobile, or some kind of hybrid. And that depends on the best way to reach the particular population that you’re studying.
How do you go about analyzing the data? Does that differ for qualitative versus quantitative data?
VR: Absolutely. (Laughing) Is this interesting to people?
SP: Sure! You don’t have to get in the weeds.
VR: We do an analysis, we’ll match the
method. Qualitative analysis, if it’s an IDI or a focus group, we’ll transcribe the responses from the interviews or the group. We start there. Look for themes.
BK: Bigger picture that applies to both you’re looking for themes, commonalities, patterns. Or, why is that person an outlier?
SP: Unique things. You’re looking for patterns and outliers.
VR: You code the data, both the qualitative and quantitative data.
SP: You organize it by topic, and then you
analyze it, according to patterns and themes and outliers. You see what are the general patterns and what are the unique things. Are those unique things, do they relate to anything, like a particular segment? Or do they relate to a particular attitude?
If we’re doing a quant study, there’s usually some kind of multivariate analysis. If you’re trying to predict what is driving something, you might do a regression analysis. If you’re trying to do a segmentation, you might do a cluster analysis. Those kinds of things. It really just depends on the study’s objectives, but in general, you’re examining things by topic and then looking for patterns and outliers.
VR: And you approach it differently for qualitative and quant. Because the data is presented differently.
SP: And you usually go into a study with some idea of particular segments that you might be interested in. So, you’re always looking at, well, is there any variation in this pattern by segment?
BK: Which would have come up prior, in methodology. We usually know segments ahead of time and have to make sure we have enough completed interviews in that group to be meaningful.
SP: Yeah, we did a study for a professional association, and they knew they were having trouble retaining young members. They wanted to understand their members in general, but also wanted to take a deeper look at the unmet needs of their young members.
It sounds like with some projects, you start with an explorative qualitative phase and then broaden out into a large-scale quantitative study. Can you think of an example where, while you’re doing the qualitative research, something unusual or unexpected came up that really did shape the quant?
SP: The qualitative always has some impact on the quantitative. There are always things that we don’t know about a market, nuances about customers that we learn from the qualitative work.
Somewhat related. We did a study for one of the nation’s major universities, and they were having trouble getting alumni engaged. We did a baseline survey and found that alumni were really happy with their experience while at the school, but they were not as happy with their experience as alumni.
So, the alumni council for the school really wanted to know why, what was the challenge with their alum experience? So, we did a deeper survey, and we found that the more engaged alumni were the least satisfied. That was a real disconnect and a revelation for our client, because they thought if they were so engaged, they would be more satisfied. But the reality is they’re the least satisfied, and it turns out it’s because the school and the administration had done very little to provide the infrastructure for their engagement.
There were alumni all over the world, and they needed best practices for pulling together alumni clubs, for pulling together fundraisers. And the school had very little infrastructure to support them.
BK: They were having to do all the work themselves.
SP: They wanted to be involved, but they had to reinvent the wheel all the time. And these are busy people. They’re all business executives, this is a leading business school.
So that was a huge revelation, and from that research, they developed an entire alumni infrastructure, including an online volunteer portal, best practices, and support for the clubs around the world, so the alumni could actually be more successfully engaged with the school.
Ok, so to wrap up, why do you recommend custom research?
VR: (Laughs) Why would you shop off the rack when you could have couture?
BK: People all the time say, “I’m just going to use Survey Monkey, I’ll get the answer.” And yes, you’ll get an answer, but it’s probably not going to be—
SP: It will be superficial, it might not be the right people.
VR: Your answers are only as good as the data you collect, which is highly dependent on who you’re talking to and the questions you’re asking.
SP: To get the right answers, you have to ask the right questions of the right people.