Insights with Impact Blog

Out and About: NEMRA Conference Goes Back to Basics with a Focus on Questions


By Sarah Tower-Richardi

At the New England Market Research Association’s (NEMRA) semi-annual conference, the Anderson Robbins Research team had the pleasure of connecting with, and learning from, our peers in the market research industry. All of the sessions were thought-provoking and centered around the theme of “questioning”.

Asking the right, well-developed questions, both before beginning a study and during the surveying phase, are crucial elements for successful marketing research projects. And so, it’s no surprise that these topics were touched upon in almost every session. Below are a few of the key take-aways on these important topics:

Asking the right questions before starting a project.
Many of the panelists stressed that the key to turning research into actionable results is to truly understand the research question you’re aiming to answer before you start a project. And this information is often learned by asking questions with key stakeholders, such as “what is the business decision you are going to make with this data?” Trying to glean valuable insights from your data without knowing this information, becomes much more difficult. This is second nature at Anderson Robbins.

Asking the right questions in a survey.
Just as important as asking good questions before starting a project, is the importance of asking quality questions during the surveying phase. As our first presenter, Ted Pulsifer said, “the data you get is only as good as the instrument [questionnaire] they [the respondents] are sent to.” This first session, on the importance of writing a good questionnaire, set the tone for the subsequent sessions and the topic was touched upon by many researchers throughout their presentations. Although we as researchers may think that we are always writing good questionnaires, it is easy to overlook some of the elements which are important to create an intuitive, thoughtful, and engaging questionnaire. David Harris explained four characteristics which are important for questions to embody. Questions should be:

1. Clear – Since you, as researchers, are not present to answer respondents’ questions as they are taking the survey, the questions need to be worded as clearly and unambiguously as possible.

2. Answerable – It’s very difficult to accurately answer a question for which you know nothing about. Ensuring that the respondent knows the information / topic matter that you are inquiring about will help to ensure that you get accurate data.

3. Easy – Tying back to the Clear point above, poorly worded and confusing questions are hard for respondents to answer, and additionally, they detract from having an enjoyable survey taking experience. Personalizing the question by wording it to ask about ‘you,’ (the respondent), is another way to make questions easier to understand and more enjoyable to answer. Additionally, personalization adds to the richness and depth of the given responses. As Frank Kelly demonstrated in his presentation, respondents were much more likely to give in-depth and thoughtful answers when the question was personalized, compared to when it was not.

4. Unbiased – Ensuring that your questions are not biased in any way is important to guarantee that your data is accurate and uninfluenced. The presenters raised caution around the use of Agree / Disagree scales as they inherently carry a positive bias. As David Harris mentioned, “people have more trouble disagreeing than agreeing”, and satisficing occurs all too regularly.Beyond the power of scales to influence respondents, Dr. Ronald Shapiro demonstrated the power of suggestivity, through question wording. In an engaging demonstration, Dr. Ronald Shapiro showed a volunteer a colorful birthday-themed bandana and then placed it over her eyes. Dr. Ronald Shapiro then asked the volunteer to tell him the color of the balloons on the bandana. At first the volunteer said that she did not remember any balloons, but when pushed and asked how many there were, without hesitation she said, “more than one and less than five”. Although it was a birthday-themed bandana and it could conceivably have balloons on it, there were none. This clearly demonstrates the power of questions to influence participants’ responses.

As researchers, it’s important for us all to consider the impact that questions have on our projects. The questions we ask and the questionnaires that we use to gather data are part of our professional discipline and should be treated as communication – as they are so important for the quality of our research.