Tuesday, April 19, 2011

Told You So... Margins of Error

Hate to say I told you so, but I did. Today, in Nanos' 'Leadership Index' Harper is up a whopping 15%, more than making up for the amount he dropped. Since the attacks ads are still running, I guess those ads weren't as effective as some thought, eh?

Sure some will say it was turned around because of the CPC's job in highlighting the Liberals attributing something to Harper that he didn't say. But journalists were decrying the Conservatives as having blundered in doing so too.

So without any likely explanation for a sudden drop in people's views of Harper followed by a sudden jump, its more reasonable to believe that their support didn't change significantly and it was just a statistical movement about the true average.

Everyone appreciates that its journalists' job to analyze the latest data and try to present a good story to explain movements in polls during an election. But lets not get carried away by single day movements within the margins of error.

Sunday, April 17, 2011

Margins of Error

Margins of error are additive.

If you were going on a trip, and were asked to measure the length, width and height of your bag to determine its length + width + height to ensure its appropriately sized. If you planned ahead you might have a tape measure with you, if not you can borrow one from someone else. Suppose your measurements are:

Length (L) = 65cm
Width (W) = 30cm
Height (H) = 120cm

To determine L+W+H you would just add the three together and say that the total L+W+H is 215cm. If one was to ask you then how accurate your measurement was, you might naively look at the smallest increment on your tape measure (for the sake of argument suppose it only measures cm), take half the smallest increment and declare that as your error. So you would report the bag as having a L+W+H as 215cm (+- 0.5cm).

Except that you would be wrong.

Each measurement you take carries with it an error of 0.5 cm because your ruler is incapable of telling you to a higher degree of accuracy than that what the that error could be compounded each time you take a measurement. If the bag was actually 120.3cm tall, you would still report it as 120cm. Likewise, if your width was actually 29.6cm, you'd only be able to see it as 30cm. So really what your measurements are is

L = 65 (+-0.5) cm
W = 30 (+- 0.5) cm
H = 120 (+- 0.5) cm

And since errors are additive you would have to say that your bag has a L+W+H of 215 (+-1.5) cm.

Great you say, but I'm never going to report the measurement error on my luggage at an airport. True, but if you were say, analyzing some polling results, this knowledge might come in handy. Because the "Leadership Index" is calculated by adding three separate values determined through asking the same people three separate questions. Ignoring the obvious problem of dependency (a voter who doesn't trust Harper is more likely to also say he's not competent and that he lacks a vision for Canada), there's also the additive error problem.

The result for each question carries with it an error of +- 5%, so if you add "Trust" + "Competency" + "Vision" together then the final result would have an error of a whopping +-15%. So if Harper was to drop say 13% that would still be within the margin of error. Any analysis of the results should therefore be extremely cautious, especially when Layton dropped 10% over the same period for no obvious reason.

In the fast-paced media world today I can understand that some journalists might jump to conclusions so that they can make a nice headline. But lets not pretend that this is factual. Or good journalism.

Saturday, April 16, 2011

Polling Trends - The Rise of Layton?

Once I again I am delinquent in posting an update, mostly because nothing has changed. The Tories are stuck around 39%, the Liberals around 27% and NDP around 19%. The debates had no significant effect, the mini-scandals have had no effect, nothing has had any effect. Or have they?

I like how some polling groups try to look at underlying perceptions of politicians (ie Nanos' leadership indicators; trust, competence, vision). It kind of gives an indication why voters might not be responding to particular attacks and why they might. Case in point is Ignatieff; who has made quite a big deal attacking Harper on the issue of 'trust' without luck. Yet, Nanos indicates that voters are nearly three times more likely to say the trust Harper the most than Ignatieff. Even if the polls are not completely accurate, on a macro scale they sho
w that Harper is considered more trustworthy than Ignatieff. That would explain why Ignatieff's attacks fall an deaf ears.

Ignoring polls that show Layton rising in support, which depends entirely one which pollster you are speaking with. Nanos' leadership indicators currently show him at his highest levels ever and rising steadily. Moreover, Nanos' three day tracking polls have shown the NDP rising in support for 7 out of the last 8 days. Comparatively, the Conservatives have risen 3 of the last 8 and the Liberals have risen in only 2 of the last 8. This doesn't mean we're going to end up with a Bob Rae type scenario where the NDP pulls a rabbit out of the hat, but this could be the real trend to watch.

Figure 1: Election polls since March 27th, including trending.

Tuesday, April 12, 2011

Polling Trends - April 12th

I didn't bother posting an update for the last number of days because there really was nothing interesting going on, although rest assured I was faithfully updating the spreadsheet with the latest polls. With the leadership debate tonight and the recent leak of a draft of the Auditor Generals report on G8 spending, there might be a change in the polls in the next few days. Or there might not.

My opinion, for what its worth, is that Liberal/NDP/Bloc partisans will take the leak as affirmation of their previously held views. Conservatives partisans will point out that its just a draft and that later drafts removed the infla
mmatory language replacing it with a relatively mild request for additional transparency. Non-partisans will shrug and won't generally care since we all already knew that the government had spent over $1 billion on the G8/G20 (think of all the press coverage on the fake lake).

We'll find out better tomorrow and the next day if this scandal has really affected the Tories. Looking at the Nanos tracking poll, when polling from April 9th was included, the Tories dropped from 40.5% to 39.5% in the three-day rolling average, indicating that the Tories polled lower on April 9th than usual. Tomorrow's poll will drop the April 9th result and add April 12th. So if the Tories' support is really unaffected, we should see their support increase slightly. If they decrease slightly, that'll mean that the issue really has affected the Tories negatively.

Although after that the effect of the debate and post-debate spin will take over, so I'm not sure if it will have a long term effect one way or another.

In my polling averaging system, the Tories sit at 38.7%, the Liberals at 27.9%, the NDP at 17.3%, the Bloc at 8.7% and the Greens at 6.5%.

Wednesday, April 6, 2011

Polling Trends - April 5th

Just a quick update, only three new polls to report on, the EKOS-iPolitics poll and two Nanos polls. No significant net change to the overall standings in my poll tracker... *yawn*

My poll compilation shows the Tories at 37.9%, the Liberals at 27.5%, the NDP at 17.2%, the Bloc at 8.7% and the Green trailing at 6.8%. Remember that this is not a straight average of the recent polls but rather is a trending of all the recent polls. Since the election call, the Liberals have slowly trended upwards and the Tories have remained stagnant at about 38%, which indicates to me that Ignatieff is making some headway in winning over some votes, but hasn't convinced the vast majority of the voters to take him serious enough to consider switching.

Either that, or most Canadians are completely tuned out.

Monday, April 4, 2011

Poll Tracker - April 3rd

Four polls to include in our poll tracker; two from Nanos, one from Leger and one from Harris Decima. All have very different results and trends; Leger and Decima show a narrowing race with the Tories looking at another minority government, Nanos shows the Tories pulling away with a majority easily within reach. The net effect is no real change day over day for the Tories in our poll tracker, while the Liberals have seen some minor improvement since the election was called.

Depending on what polls you follow, either the Liberals are closing the gap or the Liberals are losing Canadians, so I'm not even going to try to analyze the results. What I will do instead is offer another graph that shows the distribution of the polls over time. Polls that cover more than one day are shown repeated over every day that the poll was taken while Nanos' tracking polls only are shown the day they are taken.

Saturday, April 2, 2011

Loan Guarantees and Subsidies

In my opinion, Stephen Harper has done the right thing proposing to grant a loan guarantee to Newfoundland so that it can develop the Lower Churchill hydroelectric project and an undersea transmission line to sell that electricity to the USA. From a political point of view though, he's walking into a minefield that pits Newfoundland (7 seats) against Quebec (75 seats).

Rex Murphy in the article links above talks about the need to redress what Newfoundlanders feel is a gross inequity that has emerged as a result of a poor deal negotiated in 1969, but only scratches the surface of a far different and more national issue. What place does the federal government have in promoting the development of certain energy sources?

The federal government has offered direct subsidies to so-called 'environmentally friendly' options through programs such as the late ecoEnergy program. It has also promoted the nuclear industry both domestically and abroad through the use of loan guarantees and by paying for the cost of maintaining AECL. But does it go too far?

One might be tempted to argue that the federal government should have no role at all in promoting any energy source, whether that be hydroelectric, nuclear or wind. As a provincial area of responsibility, the government should butt out completely. But since the electric grid does not end at a provincial boundary, and since electricity cannot be conveniently bottled, shipped and sold to buyers south of the border, it would be unfair and short-sighted to deny the federal government any role whatsoever. The question then is how much of a role should it have; should it be a referee, intervening only when provinces have grievances between themselves? Should it be a banker, providing loans to support the development of large projects such as nuclear power plants or hydroelectric projects or transmission lines?

Personally, I have no problem with the federal government acting as a referee and a banker so long as the banker anticipates receiving his money back, which in this case, they do. A loan guarantee is far less intrusive and costly than a subsidy, and will enable the development of larger projects that have higher risks but reap greater rewards. But where do we draw the line?

Poll Tracker - April 1st, 2011

There have been three new polls since I last updated the poll tracker. Two Nanos daily tracking polls and an Ekos poll. The biggest difference between the two polls is the level of support for the Green party, which tracks higher in the Ekos polls than it does in the Nanos polls. I suspect that this has something to do with the methodology used; Nanos polls do not prompt the poll takers for a party, whereas the Ekos polls do. I'm not sure which methodology is more accurate since not prompting would probably get a better idea of what parties are actually on voters' minds, but when it comes to the polling station they will be prompted when they see the ballot. Nevertheless, I won't make a judgment call on the accuracy of a polling method and will average the polls together equally (except that the Ekos poll had a larger sample size).

The Conservatives are holding steady around 38% in my averaging and have been for the entire past week, the Liberals has slowly tracked up a few percentage points but this is partly the result of that odd Nanos result three days ago that I mentioned. Tomorrow, that day's results will be factored out and the Liberal support will drop 2-4% while the NDP will rise 2-4%. I trust that the media will make a big fuss out of what essentially is just statistical noise.

A quick note on my averaging technique; in order to reduce the effect of statistical noise on the polling results I use a relatively (for lack of a better word) conservative averaging method. To determine the value for a new day, I determine the average of the polls for that day and all previous days and take a weighted average of them all, weighing the days depending on the number of respondents and how long ago those respondents were polled.

The net effect of this is to reduce the noise, but I'm not entirely comfortable with it because it assumes that the changes in political support from one day to the next will be small or non-existant. In most cases that may be true, but if there were a significant event which shifted support one way or another, my polling trending would miss it at first.