Interesting that imports from China slowed last month, at least if you're comparing just two data points. Analysts think there is good news to be found here that the economy is not slowing down (which I hope to be true).
But, the statistical “analysis” that's presented in the news media reminds me of the very practical statistical lessons taught in my favorite book, Understanding Variation
Some headlines basically presented the trade deficit data as two data points — the deficit improved, so that's good. Some analysts called it a “whopping” improvement. But if we view just those data points, we see this “trend” (note — going up is “good” here, less of a deficit).
“Two data points do not make a trend.” We can't really tell how the deficit is doing from this view — better than last month, big deal, you might say.
Some articles, including the one linked above, go one step better, pointing out that it's the smallest deficit in six months. Now that data puts things in perspective and almost becomes “information” that we could use to draw conclusions.
Looking at six data points (the raw data is found here, by the way), we see a more complete picture:
Now the “improvement” seems a little less clear. It looks like normal variation in the monthly (albeit adjusted) deficit. It looks like we are fluctuating around a mean of about $58 billion dollars. Is $3 billion better than the mean a “whopping” difference?
Looking at a more complete picture, what headline would you write seeing three years worth of data, viewing it through the control chart “lens”?
Looking at the full data set…. I draw the conclusion we're still heading in the wrong direction, that one month of “bettter than last month” data doesn't convince me that the trade deficit is improving. It was pretty stable in 2003, then around December 2003, the deficit took a sudden drop and then stabilized again for six months. In June 2004, we see another “shift in the mean” and it looks like the deficit could merely be fluctuating around that new mean.
Does this remind you of ways your organization abuses data and “statistics”? Do you hear applause because quality or on-time delivery were “better than last month?” If so, check out Dr. Wheeler's book or click “Comments”, below, to chime in.
What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.
Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.
Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation:
I once worked with an Engineering Manager who was famous for drawing a curve based on one data point. There are many in industry today that say if you can draw a curve with one data point, two points must be a trend, and three must surely move you toward Nirvana.
There is a book that I have bought several times and no longer have on my shelf that proposes the use of X bar moving range graphing to consider mangement data. As you have done with your threee year data, the technique gives context to current data.
I worked for one manager who agreed with me and if we were in control there ws no need for further reporting on that item. We set goals around variability and forecasted shifts to the median. That was an interesting part of my career.
I saw a similar article and couldn’t believe the disparity between the data and the “analysis” (quotes made with sweeping hand gesture and sarcastic face).
Even the WSJ had this graph , which interprets the data differently than I might have. If only Wheeler were required reading…