I saw this chart recently. It shows the monthly average length of stay (LOS) at a hospital:
The narrative about the chart said it was:
“continuing a trend of reduced variation and improvement in length of stay results, [down a total of] 24% past 12 months”
But is there really a statistical trend there?
Of course, I took those 13 data points and created a “Process Behavior Chart.” Actually, it's the pairing of an “X Chart” (the data) and an “MR Chart” (or the Moving Ranges). Together, it's called an “XmR Chart” (as discussed in my book Measures of Success: React Less, Lead Better, Improve More):
What does the X Chart show us?
There's no evidence of any statistically-meaningful signals. Yes, the first data point is very near the calculated “upper limit,” but it's quite possible the monthly average is just fluctuating around a stable average.
The MR Chart shows no signals, either.
With two clicks, I can add a linear trend line in Excel (the light blue dotted line):
We have to be very careful with linear trend lines. They're mathematically correct, but can be easily misleading. They're very sensitive to the first and last data points and can show a positive trend (or a negative trend) when the metric is really just fluctuating around that average.
If we remove the last few (oldest) data points, the linear trendline becomes… flat.
Hence the problem with linear trend lines.
What do we need?
Show me the data! More data points!
We would be able to make a better determination about any sort of purported “improvement” or “trend” if we had additional historical data points. This chart shows a missing history that I'd love to fill in:
Since I don't have the data (and wasn't able to get it), all I can do is make up scenarios for illustration purposes.
What if There's Seasonality and Previous Year Was Same?
What if the previous 13 data points were literally the SAME as the 13 data points originally shown?
The X Chart shows no signals. The MR Chart shows one blip of unusual variation between data points 13 and 14.
If this were the real scenario, I'd say:
- No improvement
- No trend
Maybe There Was a One-Time Shift (No “Trend”)
Maybe the organization had a major improvement event (or there was some major outside factor) that changed the system, as illustrated in this X Chart?
This would be impressive… but I'd conclude:
- BIG improvement
- No trend
The X Chart could be updated to show the shift — the change in the average and the change in the limits at that one point in time:
What if That One Month Was a Blip?
This would be a troubling scenario:
This would be “troubling” because I'd conclude, in spite of the claims:
- No improvement
- No trend
That one high data point is a “signal” in that it's above the calculated upper limit.
So we'd want to ask, “What happened?” What changed? Why was it a strong but temporary effect? How could we prevent that from happening again? Was it naturally occurring or was it the result of some policy or system change we had put in place?
The Moral of the Story?
So what should we learn from this scenario?
- Be careful about claims made in words that cannot be backed up by a chart.
- If you have just 12 or 13 data points… get more data (another 12 months)
I hope this helps. Again, you can learn and read more about Process Behavior Charts from previous blog posts, webinars, or my book, Measures of Success.
What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.
Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.
Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation:
