The talk of Alberta’s chattering classes, when they’re not talking about the  devastating second wave of COVID-19 in Alberta, is recent polling that suggests Albertans are turning against Premier Jason Kenney. The narrative of the week is that his once-vanquished foe, Rachel Notley, might even re-enter the premier’s office as soon as 2023.

Environics released a CUPE Alberta-funded poll showing the Alberta NDP seven points ahead of the UCP and in (narrow) majority government territory. Ipsos, Leger and Angus Reid all have Premier Kenney’s approval rating among the lowest in the country. And a poll from Leger from mid-November says that 59% of Albertans are dissatisfied with the province’s COVID-19 response. 

The UCP was supposed to unite conservatives so that there was no chance of a progressive government ever returning. Is Kenney’s cowardly response to COVID’s second wave going to unravel it all in a single term?

Looking at the track record of polling in Alberta and how these polls are constructed is going to yield nothing but ambiguous answers. No matter what kind of government you want, get ready for a disappointment. 

Not all polls are created equal. Many polls aren’t even created well. Few people who read polls read the fine print about how the poll was done, or have any idea whether the poll is reliable at all. Many polls that get published have sample sizes so low that even the reported margin of error can miss the mark by 4% to 9% on every number. 

For example, a poll that conducts 1,000 interviews still can produce a number that is wrong by 3.1% in either direction. One in twenty times, it won’t even be remotely close.

Out of 11 publicly reported polls conducted since the last Alberta election, only three actually even hit the 1,000-interview mark (Environics’ survey was one of them). One poll had only 304 interviews, with a margin of error of 7%. The media does a poor job of explaining how unreliable these small polls are--so people take them to be definitive.

But there are problems with Alberta’s polling that go beyond the math and into the psychological. Polls routinely undersample conservative voters. Researchers call it “shy Tory syndrome” or the “Bradley Effect.”

Today you might more accurately call it “paranoid Tory syndrome.” Conservative culture today is led by social media accounts that claim mainstream media is out to attack their values. The new conservative narratives are about shadowy forces led by Hillary Clinton, George Soros, or Justin Trudeau undermining society. This isn’t the age of Ralph Klein--it’s the age of Q-anon.

So when a pollster calls with a twenty-question interview about politics, a lot of these conservatives hang up. A behavioral difference like this adds up to a lot of inaccuracy, especially when you add it to the unreliability of small polls.

As a result, pollsters often overestimate NDP support in Canada’s western provinces and underestimate the support of conservative parties. Think back to the 2015 election: pollsters estimated the NDP would land at 42 to 45 per cent of the vote during the last week of the campaign, but they won 40 per cent. In 2019, the NDP was polled at 34 per cent to 40 per cent, but at the ballot box only got 32.7 per cent. 

So the polls mean quite a lot less than we’re told. What about Jason Kenney’s cratering approval ratings, or the awful marks the public are giving him on Alberta’s COVID response?

Well, leader approval ratings are set up and conducted the same way as the voter-intention polls, so they have all the same problems I just described. And then they suffer from one more: popularity or unpopularity doesn’t translate directly into votes. A leader can be much more popular than other leaders and still lose elections (like the Ontario NDP’s Andrea Howarth). They can also be very unpopular and still win elections (like former three-term BC premier Gordon Campbell). 

Several of the recent polls measuring premier approval are national polls and as such have small samples at the provincial level. The Ipsos poll that had Kenney at 37% approval has a sample size in Alberta of only 100 people, and a tiny sample like that gives you an incredibly shaky result. Statistically you would expect it to be off by as much as 11 points in either direction.

Screenshot from recent Angus Reid poll.

When it comes to Albertans’ dissatisfaction with Premier Kenney’s COVID response, the pollsters also aren’t asking dissenters why they disapprove. And that’s very important. Respondents who feel the COVID measures are too comprehensive are likely to be on the extreme right, and the Q crowd is certainly not going to be casting protest votes for Rachel Notley.

You just can’t read the future in these tea leaves. The data is simply not strong enough to be a basis for any real decisions. But there’s one, final, glaring problem with all of these polls, and it’s this: no matter how good or bad your numbers are, knowing them doesn’t change your situation. Elections aren’t won by knowing who’s in the lead today, they’re won by hard work--by calling, going door-to-door, fundraising, text-banking, talking to your neighbors, and building relationships in your community.

So even if it looks great, don’t get too distracted by today’s scorecard. Start thinking about the one that really counts. The real contest is the election in 2023. And if you want to pull that in one direction or the other, you’re going to need to put down the polls, pick up your phone, and get to work with people who share your values. That’s how Kenney got to where he is today--and it’s the only way you’re going to send him anywhere else. 

John Ashton studied research methods at the University of Windsor and has been analyzing polling for almost twenty years. He is a veteran campaign organizer with 28 elections’ experience. He currently serves as executive director for an industry association.