Polling Bias
All political polling is incorrect – the final result is different from that predicted in the poll. To avoid some of these errors we aggregate and average polls (see our polling index). However, the polls collectively ultimately display biases – they understate or overstate the support of each party. This statistical bias does not mean that pollsters are deliberately changing their results to benefit one team or another. Pollsters have a clear interest in being accurate. Instead, it means that pollsters asked the wrong people, asked the wrong questions or got the wrong answers.
Political polling is very difficult because of several factors:
- Polls have very low response rates
- The kind of people who want to answer polls are not representative of the voting population
- People do not give accurate responses to polling questions
- People change their minds before polling day (about who they will vote for or whether they will vote)
- Predicting the behaviour of 30 to 35 million people based on the stated opinions of 1 000 of them is always going to be tough.
The size of the bias for each general election is shown below. The calculation is based on the value of our polling index on the day before polling day and the final vote share for each party. The important points to note are:
- The size of the polling misses are not predictable – the polls overstated Labour by 2 points in 2015, then understated them by 5 points in 2017 before getting within half a point in 2019.
- Perceptions of polling failures have little relationship to actual reality. It is commonly believed that the polls were hopelessly wrong in 1992 and largely right in 1997. That clearly is not true, on both occasions the polls greatly overstated Labour but no one really noticed in 1997 as Labour were winning a thumping landslide.
- The Conservative vote share has been understated in polls frequently in recent elections (2001, 2005, 2010, 2015 and 2019) but this is most likely just random chance. No inference should be drawn from this apparent trend.
The bias in polling is presented here to show that errors in polls can be large and are impossible to predict in advance of elections. Attempts to “unskew” the polls to take account of such bias leads to worse predictions.
It should also be noted that a great challenge of election prediction in semi or non-proportional electoral systems is converting vote shares into seats. For more on this see our analysis of geographic variation in voting.
Polling Bias in England, Wales and Scotland for UK General Elections 1955-2019
A positive number means the polls overstated the party and a negative number means the polls understated them
Polling Bias in Scotland for UK General Elections 2015-2019
A positive number means the polls overstated the party and a negative number means the polls understated them