Redesign of Diagnostic Coding in Pediatrics: From Form-based to Discharge Letter Linked

by Hilco Prins, MSc, Hans Büller, MD, PhD, and Betty Zwetsloot-Schonk, MD, PhD

Abstract

Diagnostic coding after hospital discharge is mainly based on abstracting of paper medical records by medical record coders. Studies show that the quality of these data is often moderate, possibly because discharge registries play no role in daily patient care. Timely writing of discharge letters is needed to support continuity of care, at least in the Netherlands. This article describes the redesign and evaluation of diagnosis registration and discharge letter writing at a Dutch pediatric department.

Formerly, pediatricians at this department completed discharge forms. However, many forms were completed with insufficient information or not at all. Pediatricians now provide diagnoses with codes in a special heading of the discharge letter. The medical record coder checks and corrects this diagnosis heading. A list of diagnoses for pediatrics, based on ICD-9-CM, was developed and alphabetically ordered into one booklet used by pediatricians when dictating discharge letters. A reminder system for in-time writing of letters was implemented.

Since 1995, this discharge letter-linked registration has proven to be applicable in daily care. How accurately pediatricians filled in the diagnosis heading was analyzed during two periods. In 1995, 25 percent of the diagnoses were initially (before adjustments made by the medical record coder) not coded or incorrectly coded; nine percent of these shortcomings could be attributed to the pediatricians. In 1997, 67 percent of the diagnoses were initially not coded or incorrectly coded; 37 percent of these shortcomings were attributable to pediatricians. Initially, only half of the letters were written within six weeks after discharge. The correction function of the medical record coder is indispensable.

Introduction

Process, Use, and Quality of Diagnosis Registration

Diagnosis registration in hospitals usually takes place after discharge of the patient by abstracting of the paper medical record and encoding of the resulting diagnostic information by a medical record coder into ICD-9-CM or ICD-10. ^1-2 The codes are electronically recorded, primarily for statistical, policy, or reimbursement reasons. Studies that measured quality of diagnostic data in a broad domain of medicine show inaccuracies in both completeness and correctness.^3–10 These inaccuracies hamper use of these data, for example, for next patient encounters, communication about patients, and assessment of medical practice.^11–17 Most likely, when diagnosis registration is integrated with daily medical practice, data quality will improve.^18,19 Use of a computerized patient record (CPR) may provide substantial impetus, although this is not guaranteed.^20-22 CPRs are implemented in high-care environments, like neonatology and intensive care, that are characterized by technical orientation, automatic monitoring, and supportive data recording by specialized nurses. In many other departments, it will probably be years before physicians will record diagnostic information electronically.

Discharge letters play an important role in continuity of care in some countries. The specialist sends a discharge letter to the general practitioner (GP) to inform him or her about the course during admission, the treatment, and the condition of the patient. Availability of discharge letters may decrease readmission rates.²³ However, several problems are reported, especially late receipt of the letters and missing information, including admission and discharge diagnoses.^24-29

This article is based on a research project conducted in the department of pediatrics at the Academic Medical Centre (AMC) in the Netherlands. The purpose of the project was to improve the accuracy of the diagnosis registration and accelerate discharge letter writing. This article describes the redesign process, especially the involvement of the pediatricians in it, the new registration procedure, and the evaluation of the coding performance of pediatricians in the new situation.

Methods

The Problem Case: Former Diagnosis Registration at a Pediatric Department

The pediatric department at AMC, a tertiary teaching hospital in Amsterdam, has 155 beds and handles 4,500 admissions a year. In the AMC, diagnoses of admitted patients are collected and translated into ICD-9-CM and subsequently sent to the Dutch hospital discharge registry. This registry collects, for statistical purposes, data of hospital admissions on demographic and medical characteristics of patients.

Figure 1 presents an overview of the former registration procedure at the pediatric department. After each responsibility period, wherein one specialty is responsible for the total medical care for a patient, physicians collected diagnostic data in free text on paper forms. Medical record coders in a central department encoded the physicians’ descriptions and recorded the codes in the hospital information system (HIS). A previously detected problem was that the pediatricians recorded many diagnoses inaccurately, or not at all, on the paper forms. Therefore, the medical record coder customarily used discharge letters to optimize diagnosis registration; however, feedback was never given to physicians. The pediatricians already gave much attention to the structure and content of the discharge letters. After a brief notification at discharge, the letter was to be sent to the GP within six weeks, a goal that was often not achieved.

Several types of diagnoses could be collected: reason for admission, primary diagnosis, and, when applicable, secondary diagnoses, complications, and cause of injury. In case of more than one responsibility period during hospital admission, medical record coders decided which primary diagnosis was the principle diagnosis of the admission. Diagnoses were coded according to the eight-digit AMC Diagnosis Codes (ADC), a local extension of ICD-9-CM. The first digit indicates whether a code is a disease (D), visit (V), external cause of injury (E), or morphology (M) code. The last digit is a control digit. The other six digits can be used for the diagnosis itself. Thus, depending on the length of the ICD-9-CM code, one, two, or three digits are available for extension.

The medical staff of the pediatric department felt a need for improved feedback about their medical care for the purpose of quality assessment. It was felt that the reliability of routinely collected diagnostic data was insufficient; furthermore, the coding system lacked specificity. The medical staff felt they should code diagnoses on the level of detail used in daily practice. In 1993, redesign of the diagnosis registration process was begun to make the diagnosis registration complete, correct, specific, and timely enough to support assessment of medical practice.

Therefore, the following prerequisites were formulated. The registration:

1. has a role in the care process
2. is integrated into the work of pediatricians
3. does not increase the administrative workload of pediatricians
4. forms a unique collection of diagnoses for both internal and external use
5. provides ability to convert diagnoses to ICD-9-CM
6. takes place within six weeks after discharge
7. forms a basis for further automation and fits within developments toward a CPR

Diagnosis Registration Redesign

To meet the prerequisites, a project group worked out six changes in the registration process. The group consisted of a pediatrician, medical informatician, medical record coder, epidemiologist, administrative staff members, and a junior researcher. During the development, nine pediatric subspecialty groups were involved. The process from idea to implementation took two years. The six changes were:

1. Completing the form with free text was replaced by completing a special heading on the discharge letter with standardized diagnoses and their ICD-9-CM–based local codes (Figure 2). Because of the role in continuity of care, medical staff believed that a discharge letter-linked diagnosis registration would enhance completeness and correctness of the registration.

Figure 2: Example of a Completed Medical Registration Heading at the Bottom of the
Discharge Letter

2. A reminder system for in-time writing of letters was developed. Letters were to be sent to the GP, and diagnoses recorded in the HIS, within six weeks. If necessary, the pediatrician received a first reminder four weeks after discharge.
3. The discharge letter’s new style became available on the local PC network. The discharge letter with diagnosis heading might act as a first information source in case of readmission.
4. Instead of the medical record coder, pediatricians themselves encoded the diagnoses. As pediatricians know their patients, they are in a better position to choose the appropriate codes.
5. In order to support pediatricians in finding the relevant diagnosis codes, and to meet the necessary level of detail, a portable booklet was developed with a list of pediatric diagnoses in alphabetic order (Figure 3). The new list had to be a subset of the ADC. In cooperation with the pediatricians, a selection of the relevant diagnoses was taken from the ADC; within that selection, further specifications were made. Appendix A describes the procedure.

Figure 3: Part of Booklet with Diagnoses for Pediatrics in the AMC

6. The role of the medical record coder was decentralized from the medical administration department to the pediatrics department, leading to better communication between coder and pediatricians. The coder spent less time encoding and more time advising and checking.

If, for example, a junior physician writes a discharge letter under the new diagnosis registration process, the junior physician dictates the letter and selects appropriate diagnoses with codes from the booklet. Next, a secretary types the letter with diagnoses and codes in the medical registration heading. Subsequently, a senior pediatrician checks the letter. The medical record coder checks the correctness, completeness, and specificity of the diagnoses. If the letter and heading have been agreed upon, the countersigned letter is sent to the GP and a copy is inserted into the medical record. The electronic version is placed on the local PC network. The codes are stored in the HIS by the medical record coder (Figure 4).

Implementation

The new procedure was introduced, with full commitment by the head of the department, at two staff meetings. In addition, written instructions about how to complete the diagnosis heading were incorporated in the diagnosis booklet. After implementation, the medical record coder gave feedback to the pediatricians about their coding. The new system was implemented in February 1995.

Evaluation

We evaluated how accurately and when pediatricians filled in the diagnosis heading of the discharge letters. For this, completed diagnosis headings were analyzed, and interviews were held.

To analyze the diagnostic headings, the medical record coder completed an evaluation form (Figure 5) after every first version of the 276 discharge letters of patients discharged from February to May 1995. This analysis was done at an early stage of the implementation to find out where improvements had to be made. This analysis was repeated by taking a small sample of 30 discharge letters of patients discharged in June 1997. Unfortunately, because of staff problems, it was not possible to use the same time period (February to May) in 1997. The time interval between February and May 1995 was split into two periods in order to analyze whether a positive learning curve existed. Furthermore, the results of the period from February to May 1995 were compared to the results of June 1997 in order to evaluate the long-term effect. The z-test with continuity correction was used to compare the periods.

Figure 5: Items on the Evaluation Form Filled Out by Medical Record Coder

In June 1995, four months after introduction, five junior physicians, a senior pediatrician, and the medical record coder were interviewed by a research assistant (see Figure 6 for questions asked). The semi-open interviews were directed at obtaining insight into physicians’ opinions about their new role, the feasibility of the procedure, the effect that physicians and the medical record coder expected on data quality, and their opinions on which factors of the procedure contribute to this effect.

Figure 6: List of Questions Asked at the Semi-open Interviews of Physicians and Medical Record Coder

Results

Table 1 gives the distribution of the number of diagnoses in the text of the discharge letters, according to the medical record coder. The number of diagnoses per letter ranged from two to 11. Two is the minimum number of diagnoses, as “reason for admission” and “principal diagnosis” are obligatory.

Table 2 compares the findings of the first and second halves of the February to May 1995 period; also, the whole 1995 period is compared to the June 1997 period. In the second half of the February to May 1995 period, the number of diagnoses in the incorrect category decreased, and the number of diagnoses for which text or code was missing increased; the reason for the latter observation was diagnoses and codes that were rightly not found in the booklet but, according to the pediatricians, also not very relevant for the booklet. In February to May 1995, 204 of all 814 diagnoses (25 percent) found in the text of the discharge letter were, according to the medical record coder, initially coded inaccurately; 40 (5 percent) of them were incorrectly coded (20) or wrongly categorized (20)—both situations that can be attributed to the physicians—and 164 (20 percent) were missing. However, 128 codes (16 percent) were missing because they were not listed in the diagnosis booklet; the other 36 missing codes (4 percent) can be attributed to the physicians—nine times diagnosis text and code were missing, and 27 times diagnosis text was given but not in the booklet available code. In 27 cases (3 percent), the pediatricians found it important to list the missing diagnosis in the next version of the booklet, especially diagnoses in neonatology. In 1997, 77 of 115 diagnoses (67 percent) were initially not coded, 34 codes (30 percent) because they were not listed in the diagnosis booklet. Shortcomings attributable to pediatricians increased from nine percent (4 + 5 percent) in February to May 1995 to 37 percent in June 1997.

Table 3 gives results at the level of discharge letters. From February to May 1995, 143 of 276 first versions of the letters (52 percent) showed one or more shortcomings. There were hardly differences between first and second halves of this period, except for a decrease in letters sent back to the senior physician for feedback. In 1997, 27 of 30 letters (90 percent) contained missing codes; in five letters (17 percent), the registration heading was not filled in at all.

Table 4 shows that in the period from February to May 1995, each month fewer than half of the discharge letters were sent to the GP within six weeks after discharge. This time interval is equivalent for recording diagnoses in HIS. In June 1997, 28 of 30 letters (93 percent) were sent to the GP within six weeks.

From the interviews in June 1995, four months after implementation, we learned that pediatricians had complaints about the time needed to find diagnoses in the booklet and about not finding diagnoses. There was resistance to and lack of enthusiasm for the changes. Writing discharge letters is generally seen as a burden, which was made even more complicated by diagnostic coding. The reminder system resulted, in the experience of the pediatricians, in writing a letter where formerly no letter was written at all. The knowledge that a senior pediatrician would be checking their letters compelled junior physicians to fill in the diagnosis heading more seriously. However, junior physicians criticized the lack of feedback. The medical record coder sent fewer and fewer letters back to the senior pediatrician. The pediatrician felt that this practice of feedback was a burden and took too much time. Although the medical record coder thought that diagnostic data in the letter were not nearly as complete and correct as possible, she thought that the new procedure was an improvement. Formerly, many forms were not filled in at all or filled in superficially. All believed that completeness, correctness, and specificity of the diagnosis registration had increased (although not enough) as a result of the more active way the physicians dealt with the registration, the use of the booklet, and the checks. The pediatricians did not believe that embedding the diagnosis registration in the discharge letter as such contributed to the quality of the registration.

Discussion

It is important to have a high-quality diagnosis registration. As this was not the case at a pediatric department in Amsterdam, a project was initiated to improve the registration. Today, instead of the medical record coder, pediatricians themselves encode diagnoses. After discharge, diagnoses are described in standardized form with codes in a heading of the discharge letter. In order to support the pediatricians, a booklet with alphabetically ordered diagnoses was developed. This booklet contains a selection and further specification of the ICD-9-CM. A reminder system was implemented to stimulate discharge letter writing within six weeks. The role of the medical record coder shifted from encoding to checking and advising.

Completion of the heading by the pediatrician was evaluated. In the first four months, 25 percent of the diagnoses were not coded or inaccurately coded; 9 percent of these shortcomings could be attributed to the pediatricians, 16 percent to incompleteness of the diagnosis booklet. As a consequence, more than half of the first versions of the letters contained shortcomings. In the course of 1995, there was no positive learning curve despite feedback. In 1997, 67 percent of the diagnoses were not coded or incorrectly coded; 37 percent of these shortcomings could be attributed to the pediatricians, 30 percent to incompleteness of the diagnosis booklet. Consequently, almost all first versions of the letters contained shortcomings. This represented a considerable downswing. In 1995, fewer than half of the letters were written, and subsequently accompanying diagnoses recorded, within the predefined six weeks. In 1997, more than 90 percent of the letters were written within six weeks.

From interviews early after implementation, it was learned that, although the diagnosis heading was not nearly as complete, correct, and timely as possible, the new procedure was seen as an improvement. Formerly, many forms were not filled in at all or filled in superficially. The medical record coder had to complete or correct almost all forms. Unfortunately, we could not verify this, as all forms were destroyed.

The diagnosis booklet was not complete. Some diagnoses, for example, rare diseases, were intentionally not listed in the booklet. However, in an academic medical center, rare diseases occur more frequently than in nonacademic hospitals. The neonatology department did not participate in the project; consequently, relevant diagnoses were not included. After all, many children from the neonatology department are later carried over to the pediatric department.

Three reasons were advanced for the deterioration seen in 1997. First, through the lack of feedback and diminishing attention, the pediatricians became more lax with regard to the registration. Second, new junior physicians had entered the department since the introduction and were not familiar with the new registration. There was no adequate training program. Third, although the primary goal of the new registration was to enable assessment of medical practice, structural assessment of medical practice was not implemented. In 1996, a medical practice assessment project was performed.³⁰ It showed difficulties in availability and usability of electronic data and had no sequel. To guarantee a certain level of data quality, the medical record coder was emphatically instructed to check and correct the diagnostic data critically based on the text of the discharge letter.

Conforming to sociotechnical guidelines, we gave the diagnosis registration a role in daily care, integrated it with physicians’ workflow, involved users in the development, and gave careful attention to commitment and introduction.³¹ Still, the role of the pediatricians in the process was not very successful. The linkage to the discharge letter was probably not a good choice. Physicians experience writing letters as a burden and additional coding as a complication of this task. This does not support in-time writing and accurate coding. Apparently, the role of the diagnosis heading in communication is not considered very important and thus gave physicians insufficient incentives, especially in combination with a lack of consequences of low data quality. This study shows the difference between theoretical design and practical work. In practice, processes are optimized to reach short-term goals and interest.

We expected a better quality of the discharge letter-linked diagnosis registration compared to the form-based registration. This study showed that encoding and filling in the diagnoses in a special heading of the discharge letter by the pediatricians was not optimal. It is important to realize that after checking, the medical record coder corrected imperfections she found. Whether the combined effort of pediatrician and medical record coder leads to better data quality can only be tested with a before and after study in which electronically recorded diagnostic data are compared to a gold standard. This study has been performed and showed that completeness of form-based diagnosis registration was 0.51 and of discharge letter-linked diagnosis registration 0.54. Correctness was 0.65 and 0.67, respectively. ³² Completeness is the proportion of all relevant diagnosis that is recorded. Correctness is the proportion of recorded diagnoses that is true. It was concluded that the discharge letter-linked diagnosis registration did not provide a better basis for assessment of medical practice.

When Dutch hospitals change over to the ICD-10, the AMC children’s list has to be adapted. However, in doing so, the essence of the process of the new diagnosis registration will remain the same. Besides ICD, many local or disease-specific diagnosis classifications are available. Specialists prefer to use their own classifications. If conversion tables with translation to ICD codes become available, these classifications can be built into our concept. For communication with the GP, conversion of ICD codes to International Classification of Primary Care (ICPC) codes is attractive.³³ When discharge letters are sent electronically, the GP can record diagnostic data automatically in his or her information system. GPs prefer this kind of discharge summary.³⁴

The solution chosen here can be implemented within the contemporary information infrastructure of hospitals. It is a step toward the CPR, as the physician selects standardized diagnosis descriptions and the electronically recorded diagnoses get a role in the care process. A specific diagnosis list or classification is an essential prerequisite for the CPR. However, given the somewhat disappointing results, it is questionable whether our experiment is worth following.

Other Initiatives Found in the Literature

Other initiatives have been undertaken to improve the quality of diagnosis data. Cox et al. designed and tested a checklist to improve encoding of acute myocardial infarction by a medical record coder.³⁵ Unfortunately, this solution to a particular encoding problem does not provide a solution to a generic encoding problem at a pediatric department, where a broad range of diagnoses are encountered. Hohnloser et al. tested the use of a computerized browsing and encoding tool by clinicians.³⁶ The tool was part of a CPR. They concluded that the tool could increase data quality and the volume of documented data. However, the test was performed in an ICU/CCU environment, where CPRs yield profit more obviously than in most other medical departments. Delamarre et al. described an automated coding system of free-text patient discharge summaries from the field of coronary diseases into the ICD-9-CM classification.³⁷ This system seems to offer possibilities, but it cannot be implemented in the short term at a pediatric department, as the system covers only a limited domain. Besides, it is necessary to structure the discharge summary in such a way that the system can classify diagnoses as “reason for admission,” “principal diagnosis,” or “secondary diagnoses.” Moreover, it is questionable whether in the future the discharge letter will be a basis for diagnosis registration or whether the diagnosis registration will form input for the discharge letter. The latter will probably be the case.^38,39 Van Walraven and Demers showed that using a high-quality clinical database instead of reviewing medical records improves the correctness and completeness of diagnostic coding.⁴⁰ Lorenzoni et al. showed that training of medical record coders improves quality of data abstracted from the medical record.⁴¹ Arts et al. showed that training physicians in data definition and extraction is an effective way to improve quality of intensive care data, including diagnoses.⁴²

Conclusion

Within the contemporary information infrastructure of hospitals, discharge letter-linked diagnosis registration appears feasible in routine practice, but the process is not sufficient to improve the final data quality. If the physician views the diagnosis registration as having only an additive role in communication with other healthcare providers after discharge, the correction function of the medical record coder is indispensable.

Quality of diagnosis registration is difficult to manage. Continuous attention from both the physician and medical record coder seems necessary. When diagnostic data are used for medical practice assessment and therefore affect physicians, a positive effect on the quality of the data can be expected. As McKee states, “The feedback loop must be closed.” ⁴³

Notes

1. National Center for Health Statistics. “International Classification of Diseases, Ninth Revision; Clinical Modification, Sixth Edition.” Hyattsville, MD: NCHS, 2003.
2. World Health Organization. “International Statistical Classification of Diseases and Related Health Problems, Tenth Revision.” Geneva: WHO, 1992.
3. Cleary, R., R. Beard, J. Coles, B. Devlin, A. Hopkins, D. Schumacher, I. Wickings. “Comparative Hospital Databases: Value for Management and Quality.” Quality in Health Care 3, no. 1 (1994): 3–10.
4. Elkin, P.L., A.P. Ruggieri, S.H. Brown, J. Buntrock, B.A. Bauer, D. Wahner-Roedler, S.C. Litin, J. Beinborn, K.R. Bailey, L. Bergstrom. “A Randomized Controlled Trial of the Accuracy of Clinical Record Retrieval Using SNOMED-RT as Compared with ICD9-CM.” Proceedings of the 2001 AMIA Annual Symposium, pp. 159–63.
5. Fisher, E.S., F.S. Whaley, W.M. Krushat, D.J. Malenka, C. Fleming, J.A. Baron, D.C. Hsia. “The Accuracy of Medicare’s Hospital Claims Data: Progress Has Been Made, but Problems Remain.” American Journal of Public Health 82, no. 2 (1992): 243–48.
6. Gittelsohn, A., and J. Senning. “Studies on the Reliability of Vital and Health Records: I. Comparison of Cause of Death and Hospital Record Diagnoses.” American Journal of Public Health 69, no. 7 (1979): 680–89.
7. Martinez Jimenez, R., and F. Garcia Benavides. [“Hospital Morbidity Statistics: The Accuracy of the Diagnosis Reported in the Discharge Registry Book.”] Medical Clinics (Barcelona) 96, no. 20 (1991): 765–68.
8. Prins, H., H.A. Büller, and J.H. Zwetsloot-Schonk. “Effect of Discharge Letter-linked Diagnosis Registration on Data Quality.” International Journal for Quality in Health Care 12, no. 1 (2000): 47–57.
9. Quan, H., G.A. Parsons, and W.A. Ghali. “Validity of Information on Comorbidity Derived from ICD-9-CM Administrative Data.” Medical Care 40, no. 8 (2002): 675–85.
10. van Walraven, C., and S.V. Demers. “Coding Diagnoses and Procedures Using a High-Quality Clinical Database Instead of a Medical Record Review.” Journal of Evaluation in Clinical Practice 7, no. 3 (2001): 289–97.
11. van Walraven, C., R. Seth, and A. Laupacis. “Dissemination of Discharge Summaries. Not Reaching Follow-up Physicians.” Canadian Family Physician 48 (2002): 737–42.
12. Prins, H., H.A. Büller, and J.H. Zwetsloot-Schonk. “Effect of Discharge Letter-linked Diagnosis Registration on Data Quality.”
13. Iezzoni, L.I. “Assessing Quality Using Administrative Data.” Annals of Internal Medicine 127, no. 8 (1997): 666–74.
14. McKee, M. “Routine Data: A Resource for Clinical Audit?” Quality in Health Care 2, no. 2 (1993): 104–11.
15. Prins, H., F.H. Kruisinga, H.A. Büller, and J.H. Zwetsloot-Schonk. “Availability and Usability of Data for Medical Practice Assessment.” International Journal for Quality in Health Care 14, no. 2 (2002): 127–37.
16. Safran, C., and C.G. Chute. “Exploration and Exploitation of Clinical Databases.” International Journal of Bio-Medical Computing 39, no. 1 (1995): 151–56.
17. Wyatt, J. “Acquisition and Use of Clinical Data for Audit and Research.” Journal of Evaluation in Clinical Practice 1, no. 1 (1995): 15–27.
18. Hohnloser, J.H., F. Puerner, and H. Soltanian. “Improving Clinician s’ Coded Data Entry Through the Use of an Electronic Patient Record System: 3.5 Years Experience with a Semiautomatic Browsing and Encoding Tool in Clinical Routine.” Computers and Biomedical Research 29, no. 1 (1996): 41–47.
19. Grobe, S.J. “Informatics: The Infrastructure for Quality Assessment and Quality Improvement.” Journal of the American Medical Informatics Association 2, no. 4 (1995): 267–68.
20. Porcelli, P.J., Jr. “An Electronic Documentation System Increases Diagnostic Code Capture for Very Low Birth Weight Infants.” Proceedings of the 2001 AMIA Annual Symposium, 543–47.
21. Hogan, W.R., and M.M. Wagner. “Accuracy of Data in Computer-based Patient Records.” Journal of the American Medical Informatics Association 4, no. 5 (1997): 342–55.
22. Arts, D.G., N.F. De Keizer, and G.J. Scheffer. “Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework.” Journal of the American Medical Informatics Association 9, no. 6 (2002): 600–11.
23. van Walraven, C., R. Seth, P.C. Austin, and A. Laupacis. “Effect of Discharge Summary Availability During Post-Discharge Visits on Hospital Readmission.” Journal of General Internal Medicine 17, no. 3 (2002): 186–92.
24. Bolton, P., M. Mira, P. Kennedy, and M.M. Lahra. “The Quality of Communication Between Hospitals and General Practitioners: An Assessment.” Journal of Quality in Clinical Practice 18, no. 4 (1998): 241–47.
25. Christensen, K. A., J. Mainz, and E. Kristensen. “[Communication Between Primary and Secondary Sector and Its Significance for Patient Care. Evaluation Based on Admission Records and Discharge Letters of Patients Admitted Acutely to a Department of Internal Medicine.]” Ugeskrift for Læger 159, no. 48 (1997): 7141–45.
26. Jorgensen, F. S., J. Kjaergaard, and K. Blegvad. “[Evaluation of the Discharge Summary by General Practitioners. A Questionnaire Study.]” Ugeskrift for Læger 152, no. 42 (1990): 3056–59.
27. Christensen, K. A., J. Mainz, and E. Kristensen. “[Communication Between Primary and Secondary Sector and Its Significance for Patient Care.]”
28. Raval, A.N., G.E. Marchiori, and J.M. Arnold. “Improving the Continuity of Care Following Discharge of Patients Hospitalized with Heart Failure: Is the Discharge Summary Adequate?” Canadian Journal of Cardiology 19, no. 4 (2003): 365–70.
29. van Walraven, C., and A.L. Weinberg. “Quality Assessment of a Discharge Summary System.” Canadian Medical Association Journal 152, no. 9 (1995): 1437–42.
30. Prins, H., F.H. Kruisinga, H.A. Büller, and J.H. Zwetsloot-Schonk. “Availability and Usability of Data for Medical Practice Assessment.”
31. Berg, M. “Patient Care Information Systems and Health Care Work: A Sociotechnical Approach.” International Journal of Medical Informatics 55, no. 2 (1999): 87–101.
32. Prins, H., H.A. Büller, and J.H. Zwetsloot-Schonk. “Effect of Discharge Letter-linked Diagnosis Registration on Data Quality.”
33. Hofmans-Okkes, I.M., and H.Lamberts. “The International Classification of Primary Care (ICPC): New Applications in Research and Computer-based Patient Records in Family Practice.” Family Practice 13, no. 3 (1996): 294–302.
34. Brazy, J.E., D.L. Langkamp, N.D. Brazy, and R.F. De Luna. “Do Primary Care Physicians Prefer Dictated or Computer-generated Discharge Summaries?” American Journal of Diseases of Children 147, no. 9 (1993): 986–88.
35. Cox, J.L., M.P. Melady, E.Chen, and C.D. Naylor. “Towards Improved Coding of Acute Myocardial Infarction in Hospital Discharge Abstracts: A Pilot Project.” Canadian Journal of Cardiology 13, no. 4 (1997): 351–58.
36. Hohnloser, J.H., F. Puerner, and H. Soltanian. “Improving Clinician s’ Coded Data Entry Through the Use of an Electronic Patient Record System.”
37. Delamarre, D., A. Burgun, L.P. Seka, and P. Le Beux. “Automated Coding of Patient Discharge Summaries Using Conceptual Graphs.” Methods of Information in Medicine 34, no. 4 (1995): 345–51.
38. Adams, D.C., J.B. Bristol, and K.R. Poskitt. “Surgical Discharge Summaries: Improving the Record.” Annals of the Royal College of Surgeons of England 75, no. 2 (1993): 96–99.
39. Smith, R.P., and G.B. Holzman. “The Application of a Computer Data Base System to the Generation of Hospital Discharge Summaries.” Obstetrics & Gynecology 73, no. 5 (1989): 803–7.
40. van Walraven, C., and S.V. Demers. “Coding Diagnoses and Procedures Using a High-Quality Clinical Database Instead of a Medical Record Review.”
41. Lorenzoni, L., R. Da Cas, and U.L. Aparo. “The Quality of Abstracting Medical Information from the Medical Record: The Impact of Training Programmes.” International Journal for Quality in Health Care 11, no. 3 (1999): 209–13.
42. Arts, D.G., R.J. Bosman, E. de Jonge, J.C. Joore, and N.F. de Keizer. “Training in Data Definitions Improves Quality of Intensive Care Data.” Critical Care 7, no. 2 (2003): 179–84.
43. McKee, M. “Routine Data.”

Appendix A

Strategy to Develop a Portable Booklet with a List of Pediatric Diagnoses in Alphabetic Order

1. Diagnoses used at the pediatric department of the Leiden University Medical Centre from 1978 to 1992 were taken as a starting point. This set encompassed about 1,700 different ICD-9-CM-based codes. Each diagnosis code was provided with a subspecialty code.
2. A draft AMC children’s list was composed by selecting the matching ADC diagnoses. Subspecialty codes of the Leiden children’s list were also incorporated.
3. Using the subspecialty codes, the draft AMC list was divided into subspecialty lists. Every subspecialty criticized its own list with the following comments: (a) diagnosis has to be made more specific; (b) diagnosis has to be formulated differently; (c) diagnosis can be omitted; and (d) missing diagnosis has to be added.
4. The subspecialties pediatric cardiology, oncology, and genetics had their own, non ICD-9-CM-based, classifications. Diagnoses in these classifications were translated to the ADC. Where diagnoses were more specific than those of the ADC, ADC codes were extended. The translated lists were finalized with comments from the subspecialties concerned.
5. Lists from steps three and four were combined, resulting in a list of 2,480 diagnoses. In order to make it easy for pediatricians to find diagnoses, syntax rules were applied and led to clustering of diagnoses, for example, diagnoses that relate to meningitis (Figure 3). For oncology, the topography (T) codes and associated morphology (M) codes were listed separately. Overlapping and uncommon diagnoses were omitted. The booklet contained 1,560 diagnoses, of which 468 were newly specified.

Hilco Prins, MSc, is assistant professor in the department of medical informatics, Academic Medical Center, University of Amsterdam. Hans Büller, MD, PhD, is a professor in pediatrics and head of the department of pediatrics at Sophia Children’s Hospital – Erasmus Medical Center, Rotterdam, the Netherlands. Betty Zwetsloot-Schonk, MD, PhD, is a professor in medical informatics and head of the department of clinical informatics, Leiden University Medical Center, the Netherlands.