Study Shows Value of NLP in Pinpointing Quality Defects

Posted on August 25, 2011 I Written By

For years, we’ve heard about how much clinical information is locked away in payer databases. Payers have offered to provide clinical summaries, electronic and otherwise, The problem is, it’s potentially inaccurate clinical information because it’s all based on billing claims. (Don’t believe me? Just ask “E-Patient” Dave de Bronkart.) It is for this reason that I don’t much trust “quality” ratings based on claims data.

Just how much of a difference there was between claims data and true clinical data hasn’t been so clear, though. Until today.

A paper just published online in the Journal of the American Medical Association found that searching EMRs with natural-language processing identified up to 12 times the number of pneumonia cases and twice the rate of kidney failure and sepsis as did searches based on billing codes—ironically called “patient safety indicators” in the study—for patients admitted for surgery at six VA hospitals. That means that hundreds of the nearly 3,000 patients whose were reviewed had postoperative complications that didn’t show up in quality and performance reports.

Just think of the implications of that as we move toward Accountable Care Organizations and outcomes-based reimbursement. If healthcare continues to rely on claims data for “quality” measurement, facilities that don’t take steps to prevent complications and reduce hospital-acquired infections could score just as high—and earn just as much bonus money—as those hospitals truly committed to patient safety. If so, quality rankings will remain false, subjective measures of true performance.

So how do we remedy this? It may not be so easy. As Cerner’s Dr. David McCallie told Bloomberg News, it will take a lot of reprogramming to embed natural-language search into existing EMRs, and doing so could, according to the Bloomberg story, “destabilize software systems” and necessitate a lot more training for physicians.

I’m no technical expert, so I don’t know how NLP could destabilize software. From a layman’s perspective, it almost sounds as if vendors don’t want to put the time and effort into redesigning their products. Could it be?

I suppose there is still a chance that HHS could require NLP in Stage 3 of meaningful use—it’s not gonna happen for Stage 2—but I’m sure vendors and providers alike will say it’s too difficult. They may even say there just isn’t enough evidence; this JAMA study certainly would have to be replicated and corroborated. But are you willing to take the chance that the hospital you visit for surgery doesn’t have any real incentive to take steps to prevent complications?