Did This Intervention Reduce Blood Culture Contamination Rates?

Here is another case example that uses the “Process Behavior Chart” methods from my book Measures of Success: React Less, Lead Better, Improve More.

You might ask, “Why does he keep blogging about this topic? I thought this was a blog about Lean.”

Well, for one, I'm promoting my new book :-)

Secondly, I think these methods are very important for people doing Lean improvement work. There's a risk that people might declare victory too soon when they are evaluating the impact of an attempted improvement. Does one data point prove that the intervention has improved the system significantly?

Here is a chart from a team in San Antonio, where I teach a class on Lean for quality and patient safety twice a year. This team was basically using the “A3” improvement methodology to look at a problem that really impacted patients. My understanding is that a contaminated blood culture bottle leads to false positive results that indicate that a patient has a bloodstream infection when they really don't. As it says here:

“These contaminated blood cultures may result in the administration of unnecessary antibiotics, prolonged hospital stays, and significant increases in hospital costs.”

Here is a chart the team had:

In Measures of Success, I cite the book Meaningful Graphs that makes the case that time-series data, like the chart above, should really be drawn as a Line Chart (a.k.a. a Run Chart). I agree with that, as I agree with the idea that a Column Chart draws your eye to the middle of the bar, when it's the top of the bar that matters as the data point. Plus, a Run Chart uses less ink.

Here is a simple Run Chart. I think this is more clear. The blue dashed line is the target.

Either way, we see a target of 3.0%. The color coding (and the bars) make it clear this system is not capable of reliably beating the target. Hence, the improvement project. And, thankfully, the team was looking at this in a systematic way, as I discuss in Chapter 4 of my book. Throughout the book, I try to make the case that the answer to systemic problems won't be found in asking “what went wrong last week?” when we have a predictable system with results that predictably fluctuate around an average.

The week of 5/20/2017 was green in the column chart, being a bit below the target. We can see it's below target in the Run Chart without color coding.

The team said that week was actually different because there was an intervention to the system. But, for some reason, that process change couldn't be continued. I don't know or remember the circumstances around that. It's a shame that they might have had something that could bring the average contamination percentage down but couldn't continue it. I'd hate to think it was some cost-related factor, since blood culture contamination has its costs, as well.

The last data point, also better than the target, was the first week of the new interventions they were trying in the system. Is that data point “signal” (meaning that the change had a significant impact) or “noise?”

To answer this question, I created a Process Behavior Chart, using the first 20 data points to calculate the average and the Natural Process Limits (see the spreadsheet, if you like). This is the full combination of X Chart (the data) and the MR Chart (the “Moving Ranges” being the difference between each two successful data points).

Each of those below-target data points is “noise.” There are no signals in the chart.

Had the 10/21/17 week been below the calculated Lower Natural Process Limit of 1.1%, then the team could declare victory (one rule for finding a signal is any data point outside of the Limits). It's possible the system has changed in a way that will lead a sustained lower average (something lower than the old average of 6%). It would be a different signal to see eight consecutive data points below the old average.

I would find that single data point to be encouraging. But, it's also possible that the old system could have generated that data point on its own. It's within the bounds of expected variation.

If you wondered about that 5/20/17 data point that they said was a different process, excluding that data point from the Process Behavior Chart and the calculations doesn't really change anything significantly. I wouldn't draw any different conclusions from this PBC:

What I suggested to the team, even without having a PBC handy, was that they collect more data points before drawing any before/after conclusions.

If the future data looked like this, it would be better proof of a significant improvement to the system, showing just the X Chart:

A single data point below the arbitrary target isn't what tells us that the system has changed. Our PBC evaluation rules do.

• Rule 1: Any data point outside of the limits.
• Rule 2: Eight consecutive points on the same side of the central line.
• Rule 3: Three out of four consecutive data points that are closer to the same limit than they are to the central line.

Seeing eight consecutive data points below the old average is great. We'd actually see the third type of signal sooner… the third consecutive data point there that's closer to the Lower Limit than it is to the average. That's also unlikely to happen randomly as the result of the old system. I'd conclude that the system has changed. We could calculate new Limits and see if it's a new predictable system:

This would be great to see. The average is lower and the amount of week-to-week variation has also been reduced. This means that the new Upper Natural Process Limit is now 3.2%. This means most, but not all, weeks would likely be below the 3% target — unless something else changes in the system again (and we'd use our three rules to find any signal).

I hope this methodology seems clear… and I hope it would be helpful to you and your organization.

Learn more in my book: Measures of Success: React Less, Lead Better, Improve More.

---

What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.

Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.

---

Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation: