JAMA: The hazards of evidence-based medicine--assessing variation in care

This is an interesting commentary appearing in JAMA today regarding a study in the same issue that assessed variation in care for thyroid cancer. Actually, it’s not that interesting if you worked in health services or quality improvement research. The main thrust of the article is that we need to be very careful when using care process measures as proxies for outcomes measures. When these process measures don’t line up well with outcomes, dangerous situations can ensue where health care systems chase improvement in inappropriate processes measures that result in harmed patients. This also happens to be the main argument against pay-for-performance schemes which typically rely heavily on process measures.

Why do we use process measures if they suck so bad? Because they are easy to measure using administrative databases and electronic medical records. Outcomes are far more difficult and costly to measure.

I think this is the biggest challenge facing our health care system—figuring out ways to effectively and efficiently measure quality. I also think a fair number of health services researchers and administrators, not to mention employers, would call me crazy and slam their fists on the table while shouting about costs. I don’t deny that costs are a huge problem. However, we need to figure out ways to improve quality in order to trim unnecessary costs. We can’t improve quality without measuring it. This is essentially what Atul Gawande was getting at in his “Hotspotting” article in the New Yorker. The hotspotters he profiled were finding low quality care (through careful measurement) and figuring out ways to eliminate it which improved the health of the patients and saved money.

As this JAMA commentary points out, measurement is difficult and, consequently, bringing down costs through quality improvement will be difficult. However, my hope is that widespread adoption of electronic medical records coupled with advanced research techniques using methodologies such as natural language processing will improve the situation. I just have little hope that this will happen at any scale in the near term.