How could so many pollsters get the outcome of the presidential election so wrong?
More importantly, why did so many of the red-faced pollsters not have any clue at what went wrong?
The election did have high amperage shock value for many. This included Hillary Clinton who inherited one of the most sophisticated data systems from the Barack Obama 2012 campaign.
I’ve heard some researchers suggest that the decline in land lines for telephone surveying was a large contributor or that many Trump supporters were reluctant to communicate their true support in pre-election polling. Both may have had minimum impact, but the most significant impact on this year’s polling was this:
Everyone wants the right research, but everyone wants cheap research.
And as we all know, with any marketing investment – you get what you pay for.
The truth of the matter is something that many national pollsters hate to admit – they became “fat cats” with their only focus on delivering results that only supported their highest level of profitability at the lowest cost. It’s a perfect example of how poorly managed research is not only a bad investment, but can critically affect the welfare of your client.
What Went Wrong
From a national perspective, most pollsters would suggest that the 3-4 percent point Clinton win prediction was realized if you look at the popular vote totals. This points to poorly managed research – collecting data and not collecting accurate insight.
1. RELYING ONLY ON DATA MODELING – In the ideal world, we can construct modeling from our marketing data to make both strategic and tactical decisions. Some of the most successful brands are managed from a fully-developed and sophisticated model. Others are less successful with modeling since they don’t understand that the market is constantly dynamic and numerous data sources are needed to measure and evaluate market movement. Modeling can quickly become outdated if not supported by solid and varied market research. Also keep in mind that previous behavior doesn’t always equal predictive behavior.
2. LACK OF RESEARCH SYNTHESIS – Today’s successful marketers understand the importance of synthesizing research methodologies to validate all research findings. When marketers seek the lowest cost research investment, they turn to the cheaper alternatives. Pollsters, as well as some marketers rely heavily on singular methodologies such as online surveying or panels. Smart research plans include both quantitative and qualitative methodologies which automatically validate findings. If pollsters would have paid as much attention to the size and enthusiasm of Trump rallies, they may have questioned their quantitative findings.
3. QUALITY OF SAMPLE – Pollsters, as well as market researchers, are focused on statistical significance of their surveying. The norm is to achieve a +/- 4 percentage point maximum variance.
This goal is achieved through quantity, not quality. Unfortunately, many pollsters utilize online or panel surveying, which has duplicate participants for each research wave. In addition, because of the ease of access and availability, these methods are concentrated on high urban areas with little representation of rural citizens. In addition, representative pollster samples are developed to mirror the general population with state-by-state breakouts – not weighted for Electoral College segmented populations.
4. THE RIGHT QUESTIONS DELIVER THE RIGHT ANSWERS – As with any market research, asking the correct questions lead to obtaining the correct answers. One typical question used by pollsters, “If the election were held tomorrow, whom would you vote for?” only provides hypothetical input. It doesn’t determine who will win votes on the actual election day or even determine if the participant is committed to voting on election day.
What Can Be Learned from Failed Polling
The 2016 Campaign Poll failure only endorses what smart marketers and researchers already know in terms of quality market research:
• Relying only on behavioral data models for strategic and tactical marketing decisions has high risk. Nearly all markets are dynamic and models need to be supplemented and updated by continuous market research.
• A properly executive market research plan will include a battery of research methods – both quantitative and qualitative. Through this approach, all research analytics can be validated on a constant basis.
• Online surveying and paneling should be used with caution. Though these types of surveys can provide statistical significance, they may not provide the correct representation of your market that only traditional research methods can obtain.
• Ask the right questions and you will obtain the right answers. In today’s fast-paced marketing world, marketers trend towards quick answers versus correct answers. Survey content is as critical as the methodology or profile of the sample.
