We always look for ways to provide our clients with deeper data-driven insights – while also looking out for their bottom line. To that end, we now offer a hybrid phone and online polling approach that allows us to conduct additional in-depth analyses and provide our clients with added meaningful results.

What we do

Typically, when conducting a poll, we start with the industry’s “gold standard” approach – polling via probability driven phone sampling using professional interviewers.  This time-honed methodology is our standard and produces highly accurate results. However, by simultaneously conducting the poll online using representative samples gained through a newly-created marketplace of online panels, we’re able to increase the sample size on key questions for a relatively minor additional investment. Our online panel partner aggregates more than 200 separate online panels into a single marketplace, which allows us to reach a greater number of potential respondents across a wide variety of panels at a lower cost than traditional online panels. Thus, by layering together the phone and online data, and using the phone poll for calibration, we can double the sample size on key questions while only increasing the project budget by 10% – 20%.

A question that is often asked is, why not just use online sample since it is so much less expensive than phone? In fact, in some cases we do now recommend pure online samples.  But because of the self-selection bias inherent to online research, the inability to generate random samples, and the inability to draw samples from many discrete geographic zones, we do not believe online samples alone are dependable enough for many studies, especially political studies when we sample off voter files.  At this point, we typically suggest using online samples to supplement the phone surveys, not to supplant them.

The added value

Consider a recent study of Massachusetts voters in which we conducted 600 telephone interviews with respondents randomly selected from a statewide voter file, and then supplemented that with an additional 600 online interviews.

Using just phone sample, we could break our data into three Gender by Age groups: men under 50, men 50 – 65, men 65+, and women in these same age groups. With the additional online interviews, we could further separate the data and examine five distinct age breaks within each gender: Under 35, 35 – 44, 45 – 54, 55 – 64, 65+. This allows us to provide more targeted analysis and notably, these breaks include the crucial under-35 age group.

The larger sample size also allowed us to do entirely new analyses based on ethnicity. For example, we had just 71 interviews with non-whites in our phone sample, making any analysis by ethnicity shaky. But adding in the online sample allowed us to look at black and Hispanic voters, and to compare whites to non-whites by a variety of demographic splits, such as education, gender, income (below and above $60K), and age (under and over age 45).  These splits are valuable – and one that really stands out is comparing the attitudes of non-college minorities to those of the non-college white voters, who were such an important demographic for Trump during the presidential election.

Layering online data on top of phone data – and using gold standard methodologies to ensure quality and data accuracy – is a valuable way for us to provide our clients with deeper data-driven insights, with only a slight cost increase. In the end, it gives us a chance to provide that additional impact we’re always looking for.

The post New Offering: Finding deeper data insights through hybrid phone and online polling appeared first on Anderson Robbins.

At Anderson Robbins, we pride ourselves on our industry-leading approaches to data collection, ensuring the data we produce for our clients is as accurate as can be. But we also know that a long page or Excel sheet full of numbers doesn’t always yield the impact the data deserves.

Take, for example, our latest poll for Fox News. We asked voters to describe how well several words and phrases apply to President Trump – and here are those results, in table form.

How well do you think each of the following words or phrases describes Donald Trump extremely well?

Says what he thinks – 65%

Bully – 39%

Unstable – 33%

Outsider – 30%

Competent – 21%

Strong leader – 21%

Problem solver – 19%

Knowledgeable – 16%

Steady leader – 16%

Honest – 16%

Compassionate – 15%

Moral leader – 14%

Presidential – 13%

The table above tells the story well enough, but we can add impact in a couple of ways: with better data visualizations and by getting under the hood of the numbers.

First, we can translate the numbers above to a word cloud – a fairly basic type of visualization in which each word or term is sized relative to the frequency with which respondents said it describes Trump “extremely well.” Given the nature of the descriptors we tested, we’ve also colored the terms (green for positive; yellow for neutral; red for negative).

The visualization does a nice job of emphasizing “says what he thinks” as the best-fitting descriptor, and the two negative descriptors pop a bit as well. But here’s what happens when we dive under the hood.

Among Democrats, you need a microscope to find the positive terms, while “unstable” and “bully” are boldface criticisms of the president. Trump fares a bit better among Independents, but the neutral and negative terms continue to outstrip the positive ones. And of course, there’s a good deal of green in the GOP word cloud.

There are stories and insights hidden in every data set, every huge spreadsheet full of numbers. Our passion: bringing them out in a meaningful, impactful way.

The post Data Visualization: Looking at our Fox News Poll in a New Way appeared first on Anderson Robbins.

It’s crowded out there in the media landscape. Our eyeballs and brains are exhausted by what we read and see on traditional platforms and social media. If you want to make news, change minds or influence people it’s said that a picture is worth a thousand words. If that adage is true, I would argue that a data point is worth ten-thousand.

This summer, we helped one of our long-time partners the American Civil Liberties Union (ACLU) of Massachusetts, launch their “What a Difference a DA Makes” campaign. Using our data as the basis for change, the ACLU developed this initiative to educate voters about the powers of district attorneys, and how voters can hold prosecutors accountable and encourage them to be fair and just.

We worked with the ACLU to find out what Massachusetts voters think of the State’s criminal justice system. Only 48 percent of those surveyed said the system is working, while 73 percent – including roughly equal numbers of Democrats, Republicans and Independents – said the system works differently for different people. The dissatisfaction with the status quo resonated with reporters who used the data to write headlines and put meat onto the bones of their stories.

Whether it is criminal or social justice, the environment or bettering the education system, at Anderson Robbins Research we take pride in our passion for issues that mean something to us. It isn’t easy to help influence opinions in these areas, but it is our mission to help organizations do it because of our ability to both understand data and how to spread it.

The post Data Dissemination: ACLU & Anderson Robbins Partner to Improve the Criminal Justice System appeared first on Anderson Robbins.

At the New England Market Research Association’s (NEMRA) semi-annual conference, the Anderson Robbins Research team had the pleasure of connecting with, and learning from, our peers in the market research industry. All of the sessions were thought-provoking and centered around the theme of “questioning”.

Asking the right, well-developed questions, both before beginning a study and during the surveying phase, are crucial elements for successful marketing research projects. And so, it’s no surprise that these topics were touched upon in almost every session. Below are a few of the key take-aways on these important topics:

Asking the right questions before starting a project.
Many of the panelists stressed that the key to turning research into actionable results is to truly understand the research question you’re aiming to answer before you start a project. And this information is often learned by asking questions with key stakeholders, such as “what is the business decision you are going to make with this data?” Trying to glean valuable insights from your data without knowing this information, becomes much more difficult. This is second nature at Anderson Robbins.

Asking the right questions in a survey.
Just as important as asking good questions before starting a project, is the importance of asking quality questions during the surveying phase. As our first presenter, Ted Pulsifer said, “the data you get is only as good as the instrument [questionnaire] they [the respondents] are sent to.” This first session, on the importance of writing a good questionnaire, set the tone for the subsequent sessions and the topic was touched upon by many researchers throughout their presentations. Although we as researchers may think that we are always writing good questionnaires, it is easy to overlook some of the elements which are important to create an intuitive, thoughtful, and engaging questionnaire. David Harris explained four characteristics which are important for questions to embody. Questions should be:

1. Clear – Since you, as researchers, are not present to answer respondents’ questions as they are taking the survey, the questions need to be worded as clearly and unambiguously as possible.

2. Answerable – It’s very difficult to accurately answer a question for which you know nothing about. Ensuring that the respondent knows the information / topic matter that you are inquiring about will help to ensure that you get accurate data.

3. Easy – Tying back to the Clear point above, poorly worded and confusing questions are hard for respondents to answer, and additionally, they detract from having an enjoyable survey taking experience. Personalizing the question by wording it to ask about ‘you,’ (the respondent), is another way to make questions easier to understand and more enjoyable to answer. Additionally, personalization adds to the richness and depth of the given responses. As Frank Kelly demonstrated in his presentation, respondents were much more likely to give in-depth and thoughtful answers when the question was personalized, compared to when it was not.

4. Unbiased – Ensuring that your questions are not biased in any way is important to guarantee that your data is accurate and uninfluenced. The presenters raised caution around the use of Agree / Disagree scales as they inherently carry a positive bias. As David Harris mentioned, “people have more trouble disagreeing than agreeing”, and satisficing occurs all too regularly.Beyond the power of scales to influence respondents, Dr. Ronald Shapiro demonstrated the power of suggestivity, through question wording. In an engaging demonstration, Dr. Ronald Shapiro showed a volunteer a colorful birthday-themed bandana and then placed it over her eyes. Dr. Ronald Shapiro then asked the volunteer to tell him the color of the balloons on the bandana. At first the volunteer said that she did not remember any balloons, but when pushed and asked how many there were, without hesitation she said, “more than one and less than five”. Although it was a birthday-themed bandana and it could conceivably have balloons on it, there were none. This clearly demonstrates the power of questions to influence participants’ responses.

As researchers, it’s important for us all to consider the impact that questions have on our projects. The questions we ask and the questionnaires that we use to gather data are part of our professional discipline and should be treated as communication – as they are so important for the quality of our research.

The post Out and About: NEMRA Conference Goes Back to Basics with a Focus on Questions appeared first on Anderson Robbins.

Frustrated by employees’ inability to develop creative ideas for advertising campaigns, industry executive Alex Faickney Osborn began developing methods for creative problem-solving in 1939. He began hosting group-thinking sessions and discovered a significant improvement in the quality and quantity of ideas produced by employees.

Much like Osborn, professional moderators today have techniques to help get their respondents thinking more deeply. In focus groups sessions, your respondents are your data. What they say is crucial. But from time to time, traditional questions and probes can sometimes fail to get deep enough into respondents’ perceptions, opinions, beliefs, attitudes – or POBAs. This is because sometimes, respondents do not have a way to verbally access their POBAs.

Here are a few of my favorite alternative ways to dig deeper during a focus group session:

• Deserted Island. Describe the scenario to your respondents (have them visualize or use pictures) – Imagine, you’re being marooned by pirates to live on a deserted island, and you can only bring 10 items. What items would you bring with you, what items would you leave on the shoreline in the event you are rescued, what items would you drop into the ocean to never see again? This exercise allows the respondents to compartmentalize items/attributes (or whatever is being tested) by importance and value to them.
• Bullseye. Print-out a bulls-eye, or draw one on a large piece of paper. Stick it on the wall or an easel, and provide respondents with sticker dots. Ask a question or read a statement, then have respondents place a sticker on the bulls-eye in regards to how they feel about the statement/question or how best the statement/question describes an item, new product, etc. Allow for discussion time after. This technique will also help moderators in reporting phase to more accurately describe how a question/statement resonated with respondents.
• Happy-Crappy Scale. As moderators, we want to stay away from quantitative methods in our qualitative work. Enter, the Happy-Crappy Scale. Based on the traditional Likert Scale, the Happy-Crappy Scale shows a sad-feeling face at the bottom, and a happy face at the top, with a line connecting the two. Respondents can react to a statement, question, new product, etc. by placing a sticker dot anywhere on the scale. Allow for group discussion after. The purpose of keeping the middle of the scale blank is so that respondents don’t have to feel corralled into “feeling” a certain way. There can be many feelings between crappy and happy, and so we want to represent that in the scale.

Be sure to Follow @ARResearch for more insights into the market research industry as well as our latest news.

The post Alternative Ways to Get Below Top-of-Mind appeared first on Anderson Robbins.

Anderson Robbins is like a unicorn. It’s the mythological company you’ve always wanted to work for but thought didn’t exist. Let me tell you a little bit about why you want to join our team.

We work with some of the biggest names in business and politics and we help uncover the information they need to shape the world around us. As a part of our team, you will act as an investigative reporter discovering the most interesting data and insights from real people: Asking the right questions that drive decisions. You feel smarter when you leave work each day and you are a lot of fun at cocktail parties or playing Jeopardy on the couch.

We are looking to fill the positions of Senior Analyst; Analyst; Director and fulltime paid Intern. Check out our careers page for more details.

The post We’re Hiring appeared first on Anderson Robbins.

Making a promise just to break it after a few weeks, never seemed like much fun. But some people actually keep their New Year Resolutions. One example: Famed Facebook CEO Mark Zuckerberg. Last year, he committed to reading a new book every other week and he chronicled it on his social media platform. He also resolved to build himself a personal assistant using artificial intelligence, which he recently showed off at a Facebook Town Hall. Let’s get real. I’m not sure coding robots is achievable for me in 2017. But standing on the precipice of a new year, I asked my always motivating co-workers at Anderson Robbins Research what their resolutions are for 2017 to try and inspire my own goals. Here’s what they had to say:

Chris Anderson, President
• One thing I am committed to in 2017, is being greener. I want this to be the year that I finally get solar panels on my house.  
• In 2016, my company worked very hard to pass Question 4 in Massachusetts. I am committed to making sure the will of the voters is heard, and the measure is implemented properly in the next year.
• The political shock waves that are still reverberating around the country inspire my next resolution. I am going to remind clients and media that low probability outcomes do happen.
• Finally, on a lighter note, in 2017 I want to win at least one Pokemon battle against each of my sons. After all, low probability outcomes do happen.  

Jennifer Robbins, CEO
I’m always up for a challenge. In 2016, I chaired the March of Dimes Falmouth Road Race Team. We conquered 7.2 miles in the heat and raised thousands of dollars to make sure every baby has a fighting chance. I also decided to run my first half-marathon. While I have more races on the books for 2017, I am also going to take on my biggest challenge yet. In 2017 I am starting to write a book on the fascinating work I do everyday at Anderson Robbins Research. Stay tuned for updates on my progress.

Lauren Manganello, Director
Working fulltime and being a mom of three, I’d like to try and create more life balance in 2017. This includes practicing yoga to provide mindfulness, exercising to provide better heath and reading to expand my knowledge.

Andrew Schwartz, Director
Generally, I’m not one for resolutions, but this year I have a few goals that jump out:
• Get married. Since I’m engaged, this should be both self-explanatory and remarkably easy.
• Hike more. A major part of the reason I moved out to Denver is to enjoy the great outdoors. That was pretty challenging this fall due to the election, but it’s the best way I know to recharge.
• Apply my professional skills to local community organizations. Whether this is volunteering or bringing on new clients, I’ve got this base of knowledge and skills to help organizations advocate for a cause and communicate effectively. There are lots of local opportunities to do so – it just requires me getting out into the community and doing it. Especially important to me following the election.

Henry Kanter, Director of Operations
I’m trying to live more in the moment, not spend all my time thinking about the next moment or the moment that is five moments away. When I’m on walks, I literally stop and smell the roses. So many little things come up in my daily life that I just push past but now I’m taking a moment to check them out. I am also hoping to have the opportunity to move to somewhere closer to the ocean with my wife Cindy.

Julia Valletta, Analyst
My New Year Resolution came early! In 2017, I hoped to be able to moderate my very first in-person focus group for Anderson Robbins Research. I got that chance this past Monday in Plymouth, MN with our client, Cargill – the largest privately held corporation in the US in terms of revenue. Despite a few nervous jitters, it went smoothly… I did, after all, learn from the best – my boss, Jen Robbins.

Lauren Coates, Analyst
My New Year’s resolution for 2017 is to become more involved with charity work. Chris and Jen have been great examples of giving back to the community through their work with Read to a Child and March of Dimes.

Hannah Jaeger, Intern Extraordinaire
I’m looking forward to getting back into the habit of running, and plan to run a 5K every month. I ran my first half marathon in September and found that the benefits were beyond just the physical. When I run regularly, I find that my mind is clearer, and I am more focused and productive in my personal life as well as in my career. In fact, during my half-marathon training I received my internship position at ARR, and the race was during my first week of work! Consistent goals allow me to be the best version of myself.

As for me, there are so many things I’d like to work on 2017. I have been told by peers in the past that I thrive on a “frantic Francy” pace. Being super busy (and stressed) makes me most productive. I want to break that habit because it isn’t sustainable in the long run. My motto for 2017 will be to breathe more and talk less. I hope it will make me – and those who have to listen to me – even happier in the new year.

The post Resolution Research appeared first on Anderson Robbins.

The national polls were right. On average, national polls found Hillary Clinton up by 3 points leading up to Election Day. She appears to have won the popular vote by about 1 point.

National polls have averaged a 2-point differential from the final result over the past 12 presidential races, so the 2016 polling was about as accurate as usual.

Based largely on eight national polls I conducted in partnership with Fox News over the fall, I thought Hillary Clinton would win the national vote by 2 to 4 points – enough that an Electoral College upset would have been out of reach.

But when I revisited our pre-election polling, the story of a potentially much closer race was right in front of me. Clinton’s support had been flat and her edge ranged from just 1 to 3 points, with the exception of right after the release of the Access Hollywood video when Trump’s support briefly slipped 4 points.

So why was there such widespread surprise at a Trump victory when Clinton’s edge in polling was within the margin of error all fall?

The main reason for the great surprise was that a lower probability outcome prevailed. Donald Trump drew the inside straight many said he would need in order to prevail. Before Trump’s election, 3 of 44 presidents lost the popular vote but won in the Electoral College. The odds of drawing an inside straight are 4 in 47.
But what happened that led to this lower probability outcome?

Late deciding voters broke disproportionately for Trump.
Undecided voters and those supporting third-party candidates in our polling tended to view both Clinton and Trump unfavorably, while also thinking Clinton had the judgment and qualifications and Trump did not. I thought these voters were unlikely to break disproportionately for Trump at the end, but they did. According to exit polls, voters who were unfavorable toward both candidates voted for Trump by 20 points (49% to 29%).

Nationally, voters who decided in the final week broke for Trump, 47% to 42%.  But the late break was even more decisive in Wisconsin (59% to 30%) and Pennsylvania (54% to 37%), where Trump achieved his biggest upsets. As it turned out, late-deciding voters were more interested in “draining the swamp” than in traditional qualities associated with competence.

There was a lower and harder ceiling on Clinton’s support than many thought.
Clinton never trailed in any of the eight polls we conducted over the fall, and I assumed this meant she had a higher ceiling of support than Trump. In hindsight, I should have questioned this assumption when Clinton’s support stayed stuck at 45% after the Access Hollywood video release. When voters didn’t move toward her after the release of a video that would have been fatal to any past candidate, why would they on Election Day?

There was no Rust Belt firewall. 
Either there were some significant polling misses in Michigan, Pennsylvania and Wisconsin, or there was a huge late break for Trump. Either way, polling in these states contributed to a belief that Clinton had a Rust Belt firewall that would save her in the Electoral College, even if she lost the popular vote.

It is now clear that polling (both nationally and especially in the upper Midwest) was close enough that we should have spent more time exploring and articulating alternate outcomes rather than arguing for the data and logic that supported the higher probability outcome. But at the same time, we should recognize that when a lower probability outcome becomes the result, this isn’t necessarily a failure of polling or modeling. It might just be a lower probability outcome.

Inside straights happen, and Donald Trump just drew one.

The post An Inside Straight appeared first on Anderson Robbins.

Chris Anderson, President of Anderson Robbins Research, is the Democratic Pollster on the Fox News Decision Team. In the past two weeks, he has had the opportunity to broadcast his analysis of what the numbers mean in this ever tightening and fascinating Presidential race.

In case you missed it, here is Chris on The O’Reilly Factor and here he is On the Record with Brit Hume.

You can Follow Chris on Twitter here.

The post Chris Anderson Providing Polling Insights on Fox News Right Up to the Finish Line appeared first on Anderson Robbins.

Members of the Anderson Robbins Research team had the pleasure of attending Solomon McCown’s 24th SMC Presents panel conversation. Organized as an open-discussion, the panel was moderated by Dan Cence, Senior Vice President at Solomon McCown, and covered a range of topics concerning the intersection of government and business in Massachusetts. Here are some brief quotes and take-aways from the event:

“Government affairs is a game of addition, not subtraction.”
Matthew LeBretton, Vice President of Public Affairs for New Balance, shared this insight as a way to describe the importance of coalition building with any sort of government affairs dealing. However, he was candid in mentioning that he had to go against this principle when dealing with the recent issue between New Balance and the federal government concerning footwear for the U.S. military.

“Government relations and community relations are the same thing.”
Kathi-Anne Reinstein, Government Affairs Manager for the Boston Beer Company, said this is her approach to her job. In her eyes, what is good for the community is in turn good for the government and local businesses.

Innovation means collaboration.
Jim McGaugh is the Executive Manager of Government Relations for General Electric. He explained that GE’s organizational shift in focus to innovation led them to move from Connecticut to Massachusetts. He said the collaborative nature of government and political leaders in Boston (and Massachusetts at large) really enhanced all other aspects of the city. As McGaugh put it, “Boston had it all for GE.”

The future is in public and private partnerships.
Steve Walsh, President & CEO of the Massachusetts Council of Community Hospitals, wrapped up the panel discussion by expressing how important it is that government and business work together to build better solutions, and in turn, a better community.

If you want to learn more about Solomon McCown and their events follow them on Twitter @solomonmccown. To check out more news from Anderson Robbins be sure to follow us @ARResearch.

The post Out & About with Anderson Robbins: Backroom to the Board Room Panel with Solomon McCown & Company appeared first on Anderson Robbins.