Testing ICD-10-PCS

by Patricia E. Brooks, RRA

HCFA conducted a formal test of ICD-10-PCS (procedure coding system) in order to determine if it would be a practical replacement for the current ICD-9-CM, Procedures. This coding system was developed after an open competition contract was awarded to 3M/HIS. In order to avoid any conflict of interest, HCFA asked two other contractors to evaluate the proposed ICD-10-PCS. The contractors were two clinical data abstraction centers (CDACs)—DynKePRO in York, PA, and FMAS in Columbia, MD. The CDACs were charged with identifying necessary revisions in the current draft of ICD-10-PCS, as well as evaluating the training package with which their employees had been trained. Both contractors found ICD-10-PCS to be an improvement over ICD-9-CM, as it provided greater specificity in coding for use in research, statistical analysis, and administrative areas. A major strength of the system was its detailed structure, which allowed users to recognize and report more precisely the procedures that were performed.

After an initial training curve, the CDAC coders were able to use the system easily, with a few anticipated challenges. Because of added detail in the new system, it was occasionally necessary for the coders to utilize a medical dictionary or an anatomy textbook. Both HCFA and the CDACs recognize that coders will require a greater understanding of anatomy and surgical terms to use ICD-10-PCS than is required for ICD-9-CM. This would result in a greater amount of training time than is currently the case. Although the training manual as prepared under the contract was very useful, it needs to be strengthened with additional examples before any national training takes place. It was also suggested by the CDACs that the addition of diagrams of the body systems would be useful in the training manual.

Problems Identified

The CDAC coders quickly became proficient in the new system and were able to suggest a number of improvements, such as additional index entries and revisions to the body site and approach fields. These suggestions were forwarded to 3M/HIS for inclusion in the final draft. This exercise pointed out how easily the new system can be updated and expanded when issues are identified. Another area of concern was correct code assignment for records that did not provide enough documentation of a specific site or the type of procedure, or when the coders did not have enough knowledge of anatomy to select a precise code. Similar concerns arose about how the system would address new procedures, since ICD-10-PCS did not provide a Not Otherwise Specified (NOS)or Not Elsewhere Classified (NEC) code as is present in ICD-9-CM. Many of the questions raised by the testers after an initial period of training could have been addressed through the addition of NOS/NEC codes.

Comparison of Two Systems

A side-by-side comparison of ICD-10-PCS and ICD-9-CM was performed when the coders became proficient with the use of the new system. One contractor reported that the staff did not detect a significant time difference in using ICD-10-PCS as compared to ICD-9-CM. The other contractor found that the ICD-10-PCS process took somewhat longer. ICD-10-PCS at times required a greater number of codes than ICD-9-CM. This was due in part to ICD-9-CM's use of more combination codes than ICD-10-PCS. However, it was felt that the refined precision of ICD-10-PCS resulted in greater detail about the nature of the procedure and was therefore worthwhile. It was suggested that once coders became familiar with the greater detail and precision of the new system, the result would be improved accuracy and efficiency of coding.

Both CDAC contractors pointed out that once the coders were familiar with the new system, they rarely used the index. The ICD-10-PCS system was found to be so well organized and so well structured that coders could quickly find the correct section of the tabular list. The index was used more often when the coder was just learning the definitions of the root procedures and other basic terms used in the system. However, once coders understood the system, they found it easy to code from the tabular section.

One of the contractors pointed out that the system had only been tested on hospitalized patients. Since some hospitals currently code both inpatients and outpatients with ICD-9-CM, the CDAC suggested that additional testing should occur on outpatient records.

Major Revisions as a Result of Testing

The testers recommended a number of additions for the index and tabular sections. As a result, one of the major revisions will involve adding a NOS/NEC concept to ICD-10-PCS. As mentioned earlier, there was concern that sufficient documentation may not be present in the medical record to support the detail required by ICD-10-PCS. There was additional concern that some coders may lack the knowledge of anatomy necessary to know where a particular body part is located. To address this issue, the anatomic detail was greatly expanded in the index. To address the issue of poor documentation in the medical record, the number of approaches were reduced from 17 to 13 and an NOS/NEC concept was added to the system.

Independent Reviewers

As part of a contract awarded in 1994, the CDACs' primary task has been to collect clinical data from about 1.5 million medical records over five years. The primary end product of the CDAC contracts called for accurate and reliable clinical data in quantities sufficient to support the analytical efforts of the PROs as they carry out the Health Care Quality Improvement Program. Since the CDACs had a ready supply of current medical records and extensive experience in reviewing, abstracting, and coding medical records, they were the logical selection to act as reviewers.

Training

3M/HIS, the HCFA contractor who developed the new system, subcontracted with Rita Finnegan, RRA, CCS, a Chicago-based private consultant, to prepare an extensive training manual. The CDACs were trained for two days on the medical/surgical part of the system, and a separate one-day session was held for the remaining sections (nuclear medicine, radiation oncology, osteopathic). The CDACs then spent several weeks coding with the system to gain experience. Conference calls were held to answer questions prior to the start of the formal testing.

Methodology

In the first phase of the test, a sample of 5000 medical records (2500 per contractor) was chosen by the CDACs, identifying cases with a wide distribution of ICD-9-CM procedure codes. The CDAC coded the cases using ICD-10-PCS and noted any questions or concerns. These questions and other issues were forwarded to 3M/HIS, which then responded on an ongoing basis. As a result of this interaction, 3M/HIS made a list of revisions to be made the final draft due to HCFA in March 1998. This included terms that needed clarification and omissions in the tabular list of index sections. In addition, areas where the training manual could be improved were identified.

In the second phase of the test, a subset of 100 medical records was recoded blindly using both ICD-9-CM and ICD-10-PCS. By this time the CDAC coders were quite experienced with the new system. The reviewers began coding the first 50 records on ICD-9-CM, then moved to ICD-10-PCS. For the last 50 records, they reversed the process and began with ICD-10-PCS. The systems were to be compared on issues such as ease of use, time needed to identify codes, number of codes required, problems identifying codes, strengths and weaknesses of each system, and any other issues identified by the coding personnel.