In a NYTimes op-ed, Dr. Sandeep Juahar, the director of the Heart Failure Program at Long Island Jewish Medical Center, skewers the well-intentioned, but deeply flawed government-mandated report cards issued to practitioners in New York State, comparing their surgical outcomes to their peers. The unintended consequences of these ratings can serve only to harm patients.
Entitled Giving Doctors Grades, Dr. Juahar tells us that beginning in NY about twenty years ago and subsequently spreading (metastasizing?) to many other states, these formal evaluations (mainly for surgeons at this time) are used by state agencies to discipline providers with below-average success rates. Dr. Juahar says, Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences. The purpose of these report cards was to improve cardiac surgery by tracking surgical outcomes, sharing the results with hospitals and the public, and when necessary, placing surgeons or surgical programs on probation. The idea was that surgeons who did not measure up to their colleagues would be forced to improve. But the report cards backfired. They often penalized surgeons...who were aggressive about treating very sick patients and thus incurred higher mortality rates. When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements.
Further, he says:
The best surgeons tend to operate at teaching hospitals, where the patients are the most challenging, but you wouldn t know it from mortality statistics. It s like high school students being penalized for taking Advanced Placement courses.
In my own days in practice, I can confirm that this observation is true not only for surgeons, but for other medical specialties as well. The physicians and hospitals that handle the sickest patients are often the truly best at what they do. That s why they accept those truly challenging cases, where patients, almost by definition, end up having the worst outcomes. Conversely, hospitals and doctors that set up shop in the nicest suburbs find themselves treating the healthiest patients, who have had the resources to take care of themselves and get the best grades. This is wrong on so many levels.
Dr. Juahaar again: Report cards were supposed to protect patients by forcing surgeons to improve the quality of cardiac surgery. In many ways they have failed on this count.
Grading, quality assessment, and report cards for professionals, are one of those things that sound great on paper, but often not so much so in practice. It's simply too difficult to reliably and objectively evaluate some things.
If you haven t read Freakonomics, I strongly suggest that you do. The essence of the book is that people act in their own interest when incentivized to do so. This report card issue could easily be a chapter in the book. Bad policies will always have unintended bad consequences.