Two national presidential election polls, released this week, tell very different stories. The polls, comparing President Obama to the GOP, were conducted beginning October 6, 2011, one by NBC news and the Wall Street Journal and the other by Gallup. Although both polls had minor red flags for inaccuracy or bias, they were generally scientific and were conducted with a similar methodology. However, results relating to President Obama’s standing in the 2012 race against a Republican contender were not consistent between these polls.
Presidential Election Poll Breakdown:
The NBC poll indicated that President Barack Obama is polling at least two points ahead of all current Republican contenders, while the Gallup poll indicated that a ‘generic Republican’ is leading Obama by eight points.
Delivery of GOP vs. Obama Poll Questions
The NBC poll was conducted from October 6 to October 10, and the Gallup poll was conducted from October 6 to October 9. Both pollsters contacted respondents via landline and cellular telephones.
- Gallup: The Gallup poll had a minimum quota of 400 cell phone users and 600 landline users, with a total sample of 1,005 adults, 876 of whom are registered voters.
- NBC/WSJ poll: The NBC/Wall Street Journal poll’s respondents consisted of 1000 adults, 840 of whom are registered voters, and 200 of whom use cell phones exclusively. NBC oversampled by 81 respondents to create a separate sample of 336 Republican primary voters who were asked additional questions regarding their likely primary votes.
Oversampling is common when a poll intends to focus on a subgroup that cannot be proportionately measured within a sample meant to be representative of the nation as a whole. Weighting is used to ensure the oversample is only factored in when relevant.
Although Gallup surveyed double the amount of cell phone users as NBC, NBC states it only included cell phone users in its sample who told interviewers they used cell phones exclusively. Gallup says its survey included a mix of exclusive cell phone users, exclusive landline users, and respondents who used both, but just happened to be contacted via one or the other – Gallup used weighting to keep phone status in line with national records, and also to account for unlisted numbers, since their landline respondents were all listed in published directories.
While NBC’s method is less likely to be inaccurate due to double-counting respondents who use both types of phones, it also excludes these respondents, which may lead to inaccuracy as well. Gallup states that its sample was also weighted for gender, age, race, Hispanic ethnicity, education, region and adults in the household. There is no mention of weighting in the NBC poll results, besides the weighting that would have occurred to neutralize the oversample.
Margins of error were: +/-3.1 percent for NBC’s 1000 general interviews, +/-5.35 percent for NBC’s sample of 336 Republican primary voters, and +/-4 for the entire Gallup survey.
Wording of Poll Questions
The Gallup poll consisted only of a question asking whether the respondents are most likely to vote for Barack Obama or a generic Republican, and an additional question asking undecided respondents whether they currently lean one way or another. The NBC poll included many general questions, including some such as “All in all, do you think things in the nation are generally headed in the right direction, or do you feel things are off on the wrong track?” and “In general, do you approve or disapprove of the job Barack Obama is doing as president?” which were asked prior to the “for whom would you vote” questions. This practice tends to introduce bias against the incumbent president when the economy is at a low point, so Obama’s performance in the NBC poll may have been even stronger against the individual GOP candidates, if these questions had been excluded, or asked later in the interviews.
The ‘Generic Candidate’ Effect
Both of these polls mostly followed good scientific practices, and drew from a similar sample, yet results regarding projected 2012 voting results were dramatically different. The most logical explanation for this is that the NBC poll worded its questions so that they were about President Obama versus specific Republican candidates, and the Gallup poll referred to a generic Republican candidate.
The fact that the NBC poll has Obama beating each individual Republican candidate, but the Gallup poll has a generic Republican beating Obama by 8 points, indicates that general voters are willing to vote Republican in 2012, but none of the current candidates are able to make a significant impact against President Obama. Whether this is due to a large number of candidates splitting the Republican vote, genuine disapproval of the Republican candidates, or other factors, such as a lack of name recognition, is yet to be seen.
October Poll Results: Interpretations
Polling for the 2012 election is likely to become significantly more accurate once a Republican candidate is nominated. According to Gallup, polling with a generic candidate against an incumbent does not always produce a reliable result. This type of polling showed George H.W. Bush leading a generic Democrat in October 1991, although he ended up losing to Bill Clinton. However, George W. Bush had a lead over a generic Democrat in October 2003, and went on to win re-election.
A scenario such as this, in which an incumbent trails a generic rival in an October Gallup poll, while consistently leading specific rivals in other polls, is unprecedented in recent history.
These polls, combined with other newly-released scientific polls, support a theory that presidential approval is dwindling, but those who are disenfranchised with the president are not necessarily shifting their support to any specific Republican.
Blumenthal, Mark. Even Polls About Baseball… Pollster.com. Accessed October 28, 2011
Gallup. October 6 – 9, 2011. Accessed October 28, 2011.
NBC News/WSJ Poll. October 6 – 10, 2011. Accessed October 28, 2011.
Decoding Science. One article at a time.