Mining for Measures: NLP Technology Eases the Task of Reporting Quality Measures

by Ruth Carol

Reporting quality measures may become an easier and more fruitful task as technology’s ability to glean data from physician documentation matures.

Healthcare organizations have been asked to report quality measures since the 1990s. The activity went public when the Joint Commission began releasing such data in 2004. The Centers for Medicare and Medicaid Services (CMS) followed suit by publishing hospital data on its Web site the following year (

However, nearly two decades later the process of reporting measures remains a tedious one, still largely performed manually. This is especially true of measures collected from unstructured, free-text documents such as physician notes.

Moreover, the process is retrospective. Data are typically gleaned long after patient discharge, too late to benefit care at the point of service.

But that stands to change as technology becomes better at understanding the context of the language in transcribed or dictated physician documentation. Speech recognition coupled with natural language processing (NLP) can extract terms related to quality reporting, offering greater automation and more thorough reporting.

Asking the Right Questions

NLP can be used to abstract content from clinical documents and the electronic health record (EHR) that supports quality measure reporting, explains V. “Juggy” Jagannathan, vice president of research at MedQuist in Morgantown, WV. MedQuist combines speech recognition and NLP technology with transcription to capture clinical documentation. NLP can look not just at the words, but at their context.

Understanding the context is crucial, Jagannathan notes. It is not enough for software to locate relevant words or phrases; real benefit comes when it can reliably determine the situation and identify the care provided.

For instance, quality reporting requests several measures associated with acute myocardial infarction (AMI). CMS asks providers to report whether they: gave aspirin as soon as the patient was admitted; administered an angiotensin-converting enzyme inhibitor or angiotensin receptor blocker for left ventricular systolic dysfunction; and provided adult smoking cessation advice counseling.

Each of these measures can be viewed as a question, Jagannathan says. The NLP queries would read: “Was aspirin given as soon as the patient was admitted? Was an ACE inhibitor of angiotensin receptor blocker given for left ventricular systolic dysfunction? Was adult smoking cessation advice counseling provided?” In effect, the software combs the patient record seeking the answers.

More Documentation, Better Results

As a transcription service provider, Atlanta-based Webmedx produces physician documentation. “We use tools such as NLP technology and XQuery to search through and pull back terms related to quality measures,” explains Craig Wilkins, vice president of product management.

The underlying technology is the same that Google uses to catalogue HTML documents on the Internet. Wilkins explains, “We use a similar engine to that process that allows us to do intelligent data mining out of those documents. It is a new application of that core technology.”

In some cases, Webmedx has access to a broader document repository that includes emergency room notes, radiology reports, nursing notes, computerized physician order entry systems, or electronic medication systems. “With a more complete data set, we are able to provide more comprehensive reporting results. While the physician documentation provides the framework for all quality analysis, the other data in the EHR helps to round out the reporting picture,” he says.

Last year, Webmedx pilot tested products for use with the CMS and Joint Commission core measures among a select group of large clients. The pilot results showed a direct correlation between quality indicators obtained through manual abstraction and the same indicators measured with medical informatics tools applied to physician documentation. Webmedx plans to expand this offering in 2009 to include the full indicator set.

The Health Story Project

EHR VENDORS AND OTHERS in the industry have formed a collaborative to make text-based documentation more widely available for care and reporting. The Health Story Project is developing and promoting data standards that support the flow of information between narrative documents and EHRs. Once completed and adopted, the standards will unlock critical information from narrative documents, including the type of data that can be used for reporting quality measures.

Using the widely adopted industry standard Health Level Seven (HL7) clinical document architecture (CDA), Health Story has developed draft standards for trial use. These include standards for consultation notes as well as history and physical. Draft standards for operative notes and diagnostic imaging reports are expected soon.

Meanwhile, the Quality Reporting Document Architecture (QRDA) initiative is developing CDA standards for reporting quality measure data across vendors and disparate health IT systems that are EHR-compatible. Currently, the work is published in part as an HL7 draft standard for trial use and is being tested in pilot implementation. A standard is expected to be published this spring.

“The Health Story Project standards help bring important information produced through dictation and often needed for quality measures into the EHR, making that information more accessible and/or available to be included in QRDA reports of quality measure information,” says Joy Kuhl, director of health information technology for the Alliance for Pediatric Quality, which sponsors the QRDA initiative.

Having an EHR-compatible standard for reporting quality data will make it easier to support the analysis and tracking of healthcare quality, decrease the collection and reporting burden for providers and their organizations, and improve the quality of data used for measurement, according to Kuhl.

Health Story, formerly known as the Clinical Document Architecture for Common Document Types project, is a consortium of EHR vendors, associations, and providers. The QRDA initiative is a private collaboration sponsored by the Alliance—a joint effort of the American Academy of Pediatrics, the American Board of Pediatrics, Child Health Corporation of America, and the National Association of Children’s Hospitals and Related Institutions. For more on QRDA, see “Advancing Quality Measures Reporting in HIEs”.

From Error Reporting to “Error Catching”

The primary benefit is automation—fewer hospital personnel will have to manually review paper or electronic patient charts to determine compliance rates with quality indicators. But the technology also promises faster turnaround and more complete analysis. That could benefit patient care, reimbursement, and the organization’s own use of the data.

“This technology gets there much quicker and gives a more complete analysis,” says Wilkins. “Instead of waiting for a coding diagnosis—30 days after patient discharge—we are able to abstract an AMI case based on the patient’s history and physical, which is done within twenty-four hours of admission,” he says. “It’s as real-time as the documents are produced.” This means that physicians can use the data to focus their quality efforts during the patient’s stay.

That has major implications for quality improvement. As Nick van Terheyden, MD, succinctly states, “Error reporting becomes error catching.”

Van Terheyden is chief medical officer of Pittsburgh-based M*Modal, which also applies speech recognition and NLP to dictation. In addition to point-of-care benefits, he notes that better quality measures data can help organizations in additional ways.

Hospitals can benefit by repurposing the data depending on other needs, pulling out elements, including measures, after the fact. Just as it is the narrative that is important, not the free-form data, there is no value in just storing text, says van Terheyden. To facilitate this, M*Modal produces output in a clinical document architecture (CDA) format, an XML-based standard for the exchange of clinical documents.

A more complete analysis also helps the bottom line when revenue is tied to the public reporting of quality indictors. “The more measures the hospital can report, the higher the percentage of Medicare reimbursement it is going to get,” Wilkins says, referring to CMS’s Reporting Hospital Quality Data for Annual Payment Update program. Hospitals that do not participate in the program or do not meet its reporting requirements receive a reduction of 2 percent in their Medicare Annual Payment Update for the 2009 fiscal year.

Still Barriers

Even though companies are beginning to offer products for publicly reporting measures, there are barriers to developing a comprehensive solution.

NLP engines require more sophistication before they will abstract all measures, says Jagannathan. For providers, there is a lack of integrated health records, which fundamentally requires an EHR including dictated or transcribed physician and nursing notes or scanned documents.

“It’s about capturing various aspects of care in a systematic fashion,” Jagannathan says, “and in order to do that, the hospital needs an integrated environment of information. It’s not just a question of having the technology, it’s a question of having access to integrated health records.”

In order to use the data to affect patient care at the point of service, data must be captured while the patient is still in the hospital. Sean Carroll, CEO of Webmedx, notes, “If you can’t turn your history and physicals around in less than twenty-four hours as a regular course, the value of finding this other information in the report for measures reporting purposes is greatly diminished, if not eradicated.”

Finally, there is the ever-changing world of measures. “There are a number of organizations and standards bodies putting out competing, and sometimes conflicting, measures,” notes Wilkins. “It’s a very complicated landscape, and it’s difficult for our customers to keep up with all the metrics and reporting requirements.” As an example, this month CMS added 13 new measures and retired one of its AMI measures. Webmedx is able to stamp the logistics of those queries into the technology to accommodate such changes, he says.

“Physician documentation, principally in the form of dictation and transcription, captures the wisdom of the physician more so than almost any methodology,” concludes Carroll. “This initiative is really about cracking open that wisdom for multiple purposes, such as public reporting.”

Ruth Carol ( is a Chicago-based freelance writer specializing in healthcare.

Article citation:
Carol, Ruth. "Mining for Measures: NLP Technology Eases the Task of Reporting Quality Measures" Journal of AHIMA 80, no.4 (April 2009): 44-46.