Sunday, April 17, 2011

Margins of Error

Margins of error are additive.

If you were going on a trip, and were asked to measure the length, width and height of your bag to determine its length + width + height to ensure its appropriately sized. If you planned ahead you might have a tape measure with you, if not you can borrow one from someone else. Suppose your measurements are:

Length (L) = 65cm
Width (W) = 30cm
Height (H) = 120cm

To determine L+W+H you would just add the three together and say that the total L+W+H is 215cm. If one was to ask you then how accurate your measurement was, you might naively look at the smallest increment on your tape measure (for the sake of argument suppose it only measures cm), take half the smallest increment and declare that as your error. So you would report the bag as having a L+W+H as 215cm (+- 0.5cm).

Except that you would be wrong.

Each measurement you take carries with it an error of 0.5 cm because your ruler is incapable of telling you to a higher degree of accuracy than that what the that error could be compounded each time you take a measurement. If the bag was actually 120.3cm tall, you would still report it as 120cm. Likewise, if your width was actually 29.6cm, you'd only be able to see it as 30cm. So really what your measurements are is

L = 65 (+-0.5) cm
W = 30 (+- 0.5) cm
H = 120 (+- 0.5) cm

And since errors are additive you would have to say that your bag has a L+W+H of 215 (+-1.5) cm.

Great you say, but I'm never going to report the measurement error on my luggage at an airport. True, but if you were say, analyzing some polling results, this knowledge might come in handy. Because the "Leadership Index" is calculated by adding three separate values determined through asking the same people three separate questions. Ignoring the obvious problem of dependency (a voter who doesn't trust Harper is more likely to also say he's not competent and that he lacks a vision for Canada), there's also the additive error problem.

The result for each question carries with it an error of +- 5%, so if you add "Trust" + "Competency" + "Vision" together then the final result would have an error of a whopping +-15%. So if Harper was to drop say 13% that would still be within the margin of error. Any analysis of the results should therefore be extremely cautious, especially when Layton dropped 10% over the same period for no obvious reason.

In the fast-paced media world today I can understand that some journalists might jump to conclusions so that they can make a nice headline. But lets not pretend that this is factual. Or good journalism.

No comments:

Post a Comment