The Limited Statistical Analysis for Gallup’s Average
This spreadsheet shows the deviation for the winner of each of the 19 presidential elections covered by The Gallup Organization.
The sum total of the 19 deviations is -11.2, so the mean, or average, error is -0.59. The most naive interpretation is that the final Gallup poll is likely to understate the winner’s popular vote by just over half a percent.
The Limited Statistical Analysis for Gallup’s Standard Deviation
The standard deviation for the set of Gallup election predictions is 3.20.
The calculation is σ = ((Σ(value – mean))/(n-1))^(1/2), where:
- ‘σ’ is “lower-case sigma” in Greek, meaning the standard deviation.
- ‘Σ’ is “upper-case sigma” in Greek, meaning the sum for all instances; implicitly there are n values.
- “value[i]” is the ith value from the list.
- the “mean” or average value of all the “value[i]”.
- “/(n-1)” divides by “n-1”: divide by one less than the number of “value[i]”.
- “^(1/2)” takes the square root.
A higher value for the standard deviation indicates that the average value is less reliable.
Assuming that the errors in the Gallup results follow a “normal distribution”, one would expect that their forecast to be accurate within 3.2% in only 68% of their predictions. They should be within a 6.4% range in 95% of their predictions. (‘3.2%’ and ‘6.4%’ are “1-sigma” and “2-sigma” values).
To Analyze the Rasmussen Result
Costas Panagopoulos, PhD, ranked Rasmussen first in accuracy for the 2008 presidential election; Gallup was 17th of 20.
However, Rasmussen Reports was founded in 2003, so it is impossible to directly compare Rasmussen to Gallup; there are simply too few data points for presidential elections.
If Rasmussen was indeed absolutely accurate in 2008, it was more accurate than Gallup’s 0.1% error in 2000. The spreadsheet generously assumes that Rasmussen was equally flawless in predicting the 2004 election results.
Yet, since Rasmussen has a much shorter track record, the statistics cannot offer any assurance that Rasmussen will typically outperform Gallup.
Conclusions for the Statistical Analysis of the Reliability of Pollsters
To use statistics objectively, the two major requirements are:
- A clear question, such as “What will be the winning margin in tomorrow’s election?”
- A large, reliable, and comparable set of data points, such as predictions versus election results for several pollsters.
At this point, we can truly only review the accuracy of the Gallup polls just prior to the presidential elections; Rasmussen has made too few predictions associated with actual elections.
It would be a separate problem to compare, say, all the Gallup versus Rasmussen opinion poll results regarding Obama during 2011; but how would one determine the accuracy of each result, without output such as an actual election? Unfortunately, statistics can only help select a reliable pollster if there are reliable data points upon which to grade each pollster’s performance.
Panagopoulos, Costas (Ph.D). Poll accuracy in the 2008 presidential election: Rasmussen, Pew. (2010). Accessed May 16, 2012.
The Gallup Organization. Election Polls — Accuracy Record in Presidential Elections. Accessed May 16, 2012.
Rasmussen Reports. About Us. Accessed May 16, 2012.
Weisstein, Eric W. Standard Deviation. From MathWorld-A Wolfram Web Resource. Accessed May 16, 2012.
Decoding Science. One article at a time.