Regular readers of the blog will know that one of my favorite topics is “Gaming the Numbers” (or “Gaming the System“).
As Brian Joiner so brilliantly wrote, as shared by the late, great Peter Scholtes , there are three things people can do when faced with having to hit goals, targets, or quotas:
(1) improve the system, (2) distort the numbers, or (3) distort the system
Two items were in the news recently about #2 and #3 — from NCAA basketball and from hospitals.
Coach Cal
Kentucky head basketball coach John Calipari seems like the type of coach I'd dislike… a coach who is basically perpetuating a system of “pre-professional” players in a game that's supposed to be for “student-athletes.”
He was on ESPN recently, plugging his new book and he said some things that actually made me think of Kaizen and Lean leadership. He talked about how important it was to continually practice and to, through the course of a season, figure out the best way to incorporate different players into a system that works. It sounded like a PDSA process where they figure things out experimentally rather than just relying on throwing a bunch of individually talented people into the mix.
But, there was a story that surfaced during the NCAA Tournament that raises an ethical question – is Coach Cal distorting the system and just doing what's needed to hit a goal set by his organization, or is this dishonest?
As reported by ESPN's Darren Rovell:
Calipari used to get a $50K bonus if 75% of his senior class graduated. Often had 1 senior. UK deleted clause in 2011. (tweet)
As long as Calipari had a graduation bonus he was incentivized to always have a walk-on senior. Easy $. School realized it. (tweet)
Kentucky is known as a “one and done” school, where basketball players often just play the one season that then allows them to declare for the NBA draft. Most players don't stay all four years.
I guess 75% of zero wouldn't count toward that $50,000 bonus. So, the easy way to “game the system” is to invite a walk-on (non scholarship) player onto the team (one you know is going to graduate) and then the graduation rate was one of one or 100% –> bonus!
How'd you like to be that player who realizes that your coach is making $50,000 off of you? I'm glad Kentucky changed that bonus clause.
Gaming the VAP Numbers
I think it's generally a good trend that CMS and others are no longer paying hospitals for so-called “never events.” It's a clumsy mechanism, but the dysfunctions caused by the incentive of NOT paying for those preventable cases of patient harm is probably less than the dysfunction of the old system where hospitals got paid for the care provided to treat those preventable errors.
From the WSJ and a letter to the editor (not sure if it's behind the paywall):
The ACA inserted an economic bias when the incidence of a VAP rate was considered a quality outcome measure and could, if implemented, adversely affect a hospital's reimbursement rates. Physicians, aware of this have subtly altered the terminology for coding, substituting, for example, a diagnosis of tracheobronchitis for ventilator-associated pneumonia.
As they say, if it's easier (or perceived as easier) to distort the system than it is to IMPROVE the system… then apparently you get distortion and gaming instead of improvement.
What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.
Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.
Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation:
The creativity of human beings to work something out to their advantage in a recognition system, a bonus system, any kind of data-driven system that directly involves money in one way or another, is limitless. Trying to stop it is a fool’s errand.
The IRS is another example of such chicanery where 70% of the employees received very large (thousands of tax-payer dollars) “productivity bonuses”, and many of the recipients owed, and had not paid, back taxes. Now that is a sickness that only a wholesale shake up will have any chance of curing.
Proctor and Gamble’s “Deliberate Change” system, implemented and run by Art Spinanger, sorted it out the best I’ve ever seen. They divided recognition into “Acts of Compensation” and “Acts of Appreciation.” Their focus was on the latter. Every year they had a huge meeting of all the various improvement teams and their projects. What was vied for?? The “Golden Rock” award, which was, literally, a gold-painted rock. I confess I do not know the criteria for receiving the award but I do know it was eagerly contested for.
P&G no longer has that 30-year successful program because the new leader of it killed it by a major strategic mistake, despite it saving the company, literally, hundreds of millions every year. Same thing happened at a number of other companies with great improvement efforts.
The dictum “All is change and all is temporary” should be our guide so we won’t be disappointed when our baby is killed.
Paul
Great stories, Paul. People are VERY VERY creative when it comes to gaming the system. The problem is the system, not the people.
When organizations talk about “rewards and recognition,” they often forget how powerful honest, sincere, and public RECOGNITION is. They usually start first with rewards. I caution people against this in my Kaizen work. People don’t need bonuses or incentives to participate in Kaizen and continuous improvement.
Mark: I was leader of something called “Better Methods” for many years. Essentially Work Simplification, Material Simplification and Process Improvement. We built a recognition system around the principle: “Creating memories, not money”. Too many details to type but it was incredibly effective at recognizing the great work the foremen did by implementing Better Methods projects that improved productivity and profits in their areas of responsibility. This was from 1966 to 1986 when we expanded the concepts and Lean became the umbrella and involved all the hourly, too.
The whole effort got killed by a troglodyte VP who prevailed on a weak president to eliminate us in 1992. Such is life. We were in his way. He ruled by fear and didn’t believe in people at all. How he got hired I have no idea. So, a great company builder around Lean was eliminated.
Today, that same company has a VP charged with doing Lean and he is also head of operations, so maybe the cycle comes around again. He doesn’t really ‘get’ Lean, I don’t think, because he is focused on Kaizen events only. Sigh. I don’t hear about heavy involvement of the hourly in Lean efforts. Big Sigh.
Paul
Here is another sad example of “gaming the numbers.”
There are allegations that 40 vets have died while waiting a long time for appointments at the Phoenix VA.
Apparently, they are supposed to get appointments within 14 days of a request. But, it was apparently easier to fudge the numbers than to improve the actual system.
From the WSJ (other articles):
Shameful, but understandable if people are being pressured into a target that they can’t meet. We need to improve the system, not distort the system.
So true Mark! I have often felt “the pressure” to focus on the “system or score” versus really getting down in the weeds with process and quality improvement projects. Feels like a neverending juggling act…trying to keep my eye on all these moving parts (sometimes in vain) so the tendency is to drop all the balls and focus on the ball that will get reported. Sad, but true. The system of reporting definitely needs to change or gaming will inevitably rear its ugly head again and again!
System or the score… great comparison. Hotels and hospitals often pester guests/patients with posters and buttons that say “give us a good score!!” It’s better to actually improve the system so that you DESERVE that good score/rating.
[…] Gaming the System: Graduation Rates & Pneumonia Rates […]