Study Reveals Hard Facts on CAC

By Michelle Dougherty, MA, RHIA, CHP; Sandra Seabold, MBA, RHIA; and Susan E. White, PhD, CHDA

As the healthcare industry begins to move toward new care delivery and payment models, the importance of high quality data to support decision-making, modeling, and quality reporting has become integral to emerging systems. Diagnostic data has always been key for clinical, billing, and reporting purposes and will become even more important in supporting healthcare reform as improvements in the ICD-10-CM/PCS (ICD-10) code set are implemented on October 1, 2014.

Dr. Sam Ho, chief clinical officer at United Healthcare, noted during his keynote address at AHIMA’s 2013 ICD-10-CM/PCS and Computer-Assisted Coding Summit that the “Triple Aim” can be achieved by leveraging data-data that is improved by the ICD-10 code set. (For more on the Triple Aim, see sidebar [below].)

Better data allows providers to adhere to evidence-based medicine, analyze gaps in care, and discover disparities in appropriate use and cost, Ho said during his presentation.¹ Meeting quality measures leads to achieving part of the Triple Aim, and ICD-10 helps providers and healthcare stakeholders better measure the feasibility and accuracy of quality metrics, he said.

Understanding the importance of the transition to ICD-10 and the potential impact of computer-assisted coding (CAC) technology, the AHIMA Foundation conducted a research study in collaboration with the Cleveland Clinic to examine the interface of technology and the HIM professional. The study was funded by a research grant given by vendor 3M. Specifically, the research study examined the impact of CAC on timeliness and data quality. The pilot study sought to answer the following questions:

• Is there a measurable difference between traditional coding and the use of CAC in terms of coding timeliness and accuracy?
• Will the use of credentialed coders in conjunction with the use of CAC result in improved timeliness and accuracy?

When looking at the big picture, it is imperative that healthcare organizations plan their transition to ICD-10 with both efficiency and data quality in mind. Many organizations are evaluating CAC technology as a tool to mitigate anticipated productivity losses. Beyond efficiency, however, CAC technology should also be a tool in an organization’s information governance program. Using CAC will improve data quality with linkages to documentation improvement programs and analytics, since the technology uses natural language processing (NLP) to automatically read and translate electronic clinical documentation that suggests appropriate ICD-9/ICD-10 codes.² It was with this in mind that the AHIMA Foundation and Cleveland Clinic conducted the CAC research study.

CAC Study Process and Findings

To evaluate the timeliness and accuracy of the coding process, the study collected ICD-9 procedure and diagnostic codes on 25 Cleveland Clinic cases. Codes were assigned by 12 credentialed coders and the CAC technology. Six of the coders assigned codes without the assistance of CAC and six assigned codes with the assistance of CAC. Phase I was conducted within weeks of implementing the technology. In the second phase, conducted six months post-implementation, the 12 coders recoded the 25 records. The codes assigned by the coder and CAC were compared against the “gold standard” to assess accuracy. The gold standard is the set of correct diagnosis and procedure codes for each medical record, and was established and validated by the Cleveland Clinic coding leadership and quality team.

The sequencing of codes was not evaluated, only the presence or absence of the gold standard set of diagnosis and procedure codes. Accuracy rates were calculated through two statistical calculations: recall and precision.

Finding 1: CAC, Paired with Coders, Reduces Coding Time

Hospitals have reported that CAC has increased coder productivity by more than 20 percent, but the claim hasn’t been validated through evidence-based research.³ The AHIMA Foundation was able to validate that the time it took the study’s coders to code inpatient records using CAC was significantly shorter than those coders who didn’t use the technology, resulting in a 22 percent reduction in time per record. It should be noted that all of the 25 records coded in this study were complex with an average length of stay of 16 days and average case mix index of 2.45 (this study did not evaluate short stay or outpatient records).

The upcoming transition to ICD-10 has added new pressure on coding professionals and HIM departments to manage anticipated productivity loss. Those looking at CAC technology as a strategy to mitigate productivity loss can have increased confidence that the technology could increase throughput for inpatient cases by over 20 percent.

Finding 2: CAC, Paired with Coders, Does Not Reduce Accuracy

While the efficiency gains are important, the accuracy of the diagnostic data identified by the CAC technology is the highest priority. Cleveland Clinic has a coder training and evaluation program in which coder accuracy must be at 95 percent before they are assigned inpatient records. With such a high quality standard in place, it was important for Cleveland Clinic to not experience a reduction in quality with the implementation of CAC.

The study validated that Cleveland Clinic was able to reduce the time to code without decreasing quality as measured by recall and precision for both procedures and diagnoses. However, the study also found that CAC alone-without the intervention of a credentialed coder-had a lower recall and precision rate. The addition of a credentialed coder to the CAC improved the precision for diagnosis coding and the recall for procedure coding over using CAC alone.

Finding 3: CAC “Tuning” Improves Recall, Precision Over Time

The study tested the precision and recall performance for the CAC alone at implementation and then six months later. Since the NLP engine learns-known as tuning-over time, the CAC recall rate improved for coding both diagnoses and procedures as a result. Going forward, Cleveland Clinic could run the 25 cases through the CAC at regular intervals to evaluate improvements in precision and recall over time.

Any organization can conduct a study similar to the one performed at Cleveland Clinic to test their CAC system tuning over time. The tuning or continuous learning of the NLP engine provides an opportunity for organizations to link the use of CAC to their information governance program.

Focusing a Fuzzy Science

The coding process has always been a fuzzy science, with variability between coders. Observations in this study confirmed the variability not only between coders for the same record but by the same coder when they re-coded a record. By linking findings from documentation improvement programs and CAC tuning to priority diagnoses, such as those identified by an accountable care organization or quality reporting requirements, healthcare providers can systematically evaluate the variability and improve consistency of diagnostic data. Complete results from this study will be published in Perspectives in HIM.

Notes

1. Dimick, Chris. “ICD-10 Part of Achieving Healthcare’s ‘Triple Aim’.” Journal of AHIMA website. April 22, 2013. http://journal.ahima.org/2013/04/22/icd-10-part-of-achieving-healthcares-triple-aim/.
2. Cassidy, Bonnie. “Ten More Questions for CAC Vendors.” Journal of AHIMA website. March 1, 2013. http://journal.ahima.org/2013/03/01/ten-more-questions-for-cac-vendors/.
3. Ibid.

Michelle Dougherty (michelle.dougherty@ahimafoundation.org) is director of research and development at the AHIMA Foundation. Sandra Seabold (seabols@ccf.org) is coordinator of education, quality at the Cleveland Clinic. Susan White (white.2@osu.edu) is associate professor, clinical HRS, health information management and systems at Ohio State University.