Model for International Comparative Studies Using Routine Hospital Morbidity Data

Lauren M. Jones, MHS, Beth A. Reid, MHA, PhD, and Chris W. Aisbett

Abstract

The problem addressed in this paper is the difficulty of assessing across nations the impact of different policies on health outcomes without a reliable way of comparing routine hospital morbidity data. This paper presents the findings of a study that evaluated the accessibility and comparability of hospital morbidity data from seven nations. We present a model for comparing these data for conditions treated in three broad groups: serious emergency treatment, multiple-day stay, and multiple-day stay with a major procedure. The data were accessible, but the nations that could be compared depended on the type of clinical condition. Examples of the comparisons are presented for acute renal failure, benzodiazepine overdose, and inguinal hernia repair. We discuss the impact of differences in coding systems on hospitalisation rates and the number of codes included in the hospital morbidity data and demonstrate how these data can be used to compare clinical interest areas for the improvement and measurement of global healthcare.

Introduction

Various data sources are used to compare the health status of populations across nations, with common comparisons made from vital statistics such as birth and death registers and population censuses (OTA, 1994). Data have been standardised through WHO collaboration centers and the Organization for Economic Co-operation and Development (OECD) data collections. In addition, collaborations, such as the Cochrane group and other evidence-based groups, are using complex meta-analysis studies to compare outcomes and inform treatment practices.

The use of routine hospital data for international comparison is difficult as there are known differences in how nations define and measure disease. This is due to underlying population differences and socioeconomic factors that influence hospitalisation rates. Further, there are differences in the population representativeness of hospital data, how hospitals define admissions, and how nations classify diseases and monitor clinical indicators. There are also variances in the indications for interventions and rates at which procedures are used (OTA, 1995). However, routine hospital discharge data have been used within nations to compare severity-adjusted in-hospital death rates and invasive procedure use. See, for example, the work of Iezzoni and her colleagues (1997) on the differences between outcomes and procedure use for males and females with acute myocardial infarction or how people with disabilities experience the healthcare system (Iezzoni 2002).

Nevertheless, the question remains whether hospital morbidity data can enhance the assessment of outcomes for health policy globally. For example, evaluating patterns of the impact of wearing seatbelts on serious motor vehicle-related injuries would be feasible where hospital data from several nations could contribute comparable injury data for matching against accident data.

To use the data for international comparisons, some of the questions that need to be answered are as follows:

• Can we access the data?
• Is the geographical coverage representative of the nation (or state) as a whole?
• Is the hospital coverage (public and private) comparable?
• Are the types of patients and the hospital admission practices comparable? If different, can they be adjusted for? For example, are day cases included in the collection, how are the "dead on arrival" cases counted, and where is the boundary between hospital-based emergency services (and other ambulatory services) and hospital inpatients drawn?
• Does the content (data elements) of each data set match sufficiently for comparison, and are the classification systems similar and mappable? Are there quality issues that may affect comparison?
• What additional data are available to correct for bias in the data? For example, the OECD data may supply national indicators for health systems only partially captured in the available data.

The research project report here began as an Australian study, commissioned by a non-government organisation that evaluated hospital discharge data for specific conditions. The successful outcome of that project lead to our interest in whether other nations could be used to report outcomes similar to those reported in the Australian study and further, if the data would be suitable for a larger scale retrospective comparative study. As a result, a feasibility study was undertaken to review the accessibility of morbidity data from several nations and to develop a generalisable model for comparing such data for clinical interest areas.

The study focussed on nations with which the company shared a business interest, and thus, some nations or countries with robust morbidity systems were not included. The conditions under study were specific but in order to maintain confidentiality we made our enquiries using generic clinical focus groups. The work was undertaken between May and September 2002 and reported in 2003.

A key point to remember is that the health systems and data collections are continuously changing in many of the nations that we reviewed and more broadly. For example, we know that Ireland is currently introducing the Australian Modification of the International Classification of Diseases 10 th Revision (ICD-10-AM) as the national coding classification. Furthermore, several nations, such as Sweden and Denmark, are including more private hospitals in their morbidity collections, which may impact positively on comparability.

Method

We evaluated data collections from these seven nations: Australia, Denmark, Ireland, New Zealand, Sweden, the United Kingdom (UK) and the United States of America ( USA). We surveyed the data custodians and collected documentary evidence to gather information on the data sets. No sample data were used to verify our conclusions, as that will be done in a further project.

The first step involved categorising the comparators into three main areas: data access, coverage, and data content as shown in Figure 1.

Figure 1. Comparators Used in Feasibility Study

We then evaluated the variables for conditions requiring serious emergency treatment, multiple-day stay, and multiple-day stay with a major procedure. This enabled us to group nations into comparable cohorts for potential study .

Discussion of Results

Access to Data

Hospital morbidity data for inpatient episodes are accessible from all the nations that we reviewed. These are obtainable nationally from Denmark, Ireland, New Zealand, and Sweden. The UK maintains a standard data set called Hospital Episode Statistics (HES); however, these data must be accessed from each country within the UK (England, Northern Ireland, Scotland, and Wales). In the USA, data are available directly from the states, and there are also national databases. One such source is the HealthCare Cost and Utilization Project (HCUP) Central Distributor, which produces both state and a sample of national data in aggregated format. We found that two HCUP data collections were most relevant for comparing morbidity data: the State Inpatient Databases (SID) and the Nationwide Inpatient Sample (NIS). Table 1 presents an example of access results for three nations.

Table 1. Data Access for Australia, Sweden, and USA

When we undertook the study in 2002, data were available up to 2000 or 2001 from all the nations we reviewed. Outpatient and emergency data are not collected in all nations, yet where established, these data sets may be useful for verifying global differences between treatment populations. Nevertheless, the inpatient collections would be the core data used for comparative work .

Geographical and Hospital Coverage  

All of the data collections reviewed were representative of the larger population of each nation (state) and visitors to the nation (state). Thus, if any person residing in (or visiting) a nation requires a hospital admission, information concerning their stay would be included in the collection. In the USA, data from HCUP will only represent some of the states that collect hospital morbidity data. This may affect the inclusion of the USA in a combined study unless the differences in population distributions and socioeconomic factors can be modeled to allow statistical correction for those regions not represented. Further, in some states in the USA, information is withheld about Medicaid (low income) patients. Including states that cover all payer types and using these data in bias correction can overcome this.

There are two notable differences in terms of hospital coverage. First, some nations do not include all private hospitals (and specialty hospitals) in their collections. Second, admission practices vary from nation to nation producing variation in treatment groups. For some clinical areas, this problem will present difficulties. The main inclusions and exclusions are listed below.

The condition under study will be important in determining how much of a problem the excluded hospitals pose. For example, serious cases of acute kidney failure, especially those where individuals require transplantation, would more likely be treated in the larger hospitals, and these are generally included in the collections. Therefore, in a country such as Wales, which does not include all private hospitals in their collection, this will have a negligible impact on capturing cases of acute renal failure, as the excluded private hospitals perform minor surgery (see Table 2). Similarly, in nations such as the USA, where specialist psychiatric hospitals and obstetric hospitals are excluded from some data sets, this would not impact substantially the study of injury and internal diseases, for example.

Table 2. Hospital Coverage for Admission for Acute Renal Failure for Four Nations

Emergency attendances are excluded in most collections, however, where individuals attend emergency departments with a more serious condition, several countries admit these cases. For example, in the USA, outpatient (including emergency) and ambulatory (day surgery cases) data are excluded from the inpatient collections, and there are payment system incentives to treat individuals who present to an emergency department on an outpatient basis. A similar situation may occur in the UK. Although medical and surgical same day cases are included in the HES data set, if an individual's treatment can be attended to in the emergency department, they are not generally admitted to hospital, although there are some exceptions. In Australia and New Zealand, day cases are usually included in the inpatient collection as well as some cases treated wholly within the emergency department. Sweden was found to be similar and admit more emergency cases. Tables 3 and 4 demonstrate the differences for emergency and day only cases.

Table 3. Comparability of Attendance for Treatment for Emergency Cases Such as Benzodiazepine Overdose across Three Nations

Table 4. Comparability of Admissions for Inguinal Hernia Repair across Four Nations

Content of Data Collections

All nations included the basic data elements needed for comparative work. The basic data needed are those required by DRG grouping systems, supplemented by external cause codes, and sufficient information to identify the episode of care and its duration. Nations varied in the elements and combination of data elements released as the result of their strategies to protect the identity of the patient, the hospital, and the doctor. Consequently, information such as date of birth, postal codes, and addresses are not released unless ethical approval is obtained. Although data custodians provided comprehensive lists of data items, in some cases it was not clear what they would release. Such details will only be apparent once data are formally requested. I n some instances, the definitions of data elements differ. For example, in the UK an episode is based on the period a patient is admitted under the care of a consultant and the length of stay in hospital (or spell) may comprise one or many "consultant" episodes. To make length of stay comparable with other nations, the spell data would need to be included in a minimum data set.

Of equal importance is the need to ensure that there is comparability between the administrative and morbidity data collected, especially where diverse sources for administrative data are used and the disease classifications vary. Two nations can be compared only if data are available for similar periods and their disease and procedure codes can be matched to identify clinically similar groups. Clinical classifications for disease and interventions differed across the nations reviewed but can be mapped (refer Table 5). The number of disease and procedure codes differed across nations but whether this affects comparability of the data will depend on the condition/intervention under study. The quality and type of coding standards were generally consistent. In Australia and New Zealand, the same coding systems and standards are used. The USA and Ireland (until the introduction of ICD-10-AM in Ireland) also use the same standards for the ICD-9-CM. Coder training programs are in place in most nations and data monitoring through audits are conducted.

Table 5. Clinical Classifications Used across Nations Included in Study

Finally, many nations use personal identifiers to link patient episodes across facilities, which may enable frequency data to be analysed. Most countries adopt indicators to ascertain data quality such as missing data checks, data edits, and ongoing auditing.

Models for Comparative Studies

Once we found that we could access the data, that the data elements were similar and mappable and gained an understanding of the coverage issues, we were able to consider how the data might be modeled for comparison.

We developed three study designs (or models) for comparing the data collections, but for reasons of space will present the results of one of these--primary analysis. In primary analysis, data from several nations are mapped to a minimum data set. For primary analysis it is necessary that the coverage of hospitals and types of patients included in each national subset is representative of the nation (state) as a whole. Furthermore, it is necessary to ascertain measurable differences (similarity) in hospital admission practices and adjust for variations in how hospital inpatients are drawn from the populations under study.

Therefore, when evaluating data for primary analysis, if national data are available, the data elements are similar, and the coding classifications are comparable, then the data can be mapped to form a combined set. Once this is achieved, the geographical coverage of populations needs to be comparable, along with hospital coverage and the types of patients and admission practices. Where these criteria are met, or adjustments can be made for gaps in the data, a primary analysis is possible. We looked at these criteria and compared the nations in our study, as shown in Table 6.

Table 6. Nations Suitable for Primary Analysis for the Three Clinical Populations

Conditions requiring multiple-day stay or multiple-day stay with a major procedure presented the most stable population for comparative work.

Strategies to Adjust Data

Although we believe primary analysis is possible for several of the nations we evaluated, it is highly likely that data will need to be adjusted in order to map the data sets appropriately. One challenge is to find sufficient information to adjust for absent data, such as where private hospitals are excluded. It may be possible to request information from private facilities to ascertain the level of missing data for the condition under study and then adjust data accordingly. In addition, data on interventions can often be verified from external publications, such as the OECD, and extrapolated.

One immediate problem with comparing data across nations is the varying numbers of disease and procedures collected. This is more likely to affect the study of minor conditions and procedures but nevertheless will need to be statistically evaluated prior to comparative work. For example, we could use the data set with greatest number of codes to model the frequency of the study code in the smallest code set, and then exclude data sets if the frequency of the required disease or procedure code is too low.

Variance in the use of coding standards may also affect the ability to map the data and will again be associated with the disease under study.

Limitations

The study focussed on the hospital inpatient morbidity data collections maintained by each nation reviewed. Details were not obtained about underlying population differences, socioeconomic factors, death rates, and life expectancy, which may influence comparability.

There was no empirical verification of data undertaken. The data matching process was conducted by evaluating documented evidence of the data elements in each data set, reviewing classification code sets and standards (where available), and obtaining details about hospital treatment populations from the literature and data custodians. Furthermore, in a few instances, where evidence was not available, we were able to make inferences from outcomes data, which provided sufficient evidence that a data element or specific codes were collected (for example, existence of a DRG system clearly denotes that standard codes/data elements are collected). However, it should be kept in mind that without the data, we were unable to identify differences between nations in terms of their data quality or their clinical coding practices.

Conclusion

The study was successful in evaluating the accessibility, availability, and content of the hospital morbidity data sets under review. The study established that it is feasible to use inpatient data reported by hospitals from several nations for review of at least some conditions in the hospitalised population. Data coverage issues will impact the suitability of each nation for comparative work. These differences are much more relevant when considering conditions treated on an emergency basis, and the context in which the data are analysed by each nation would need to be explained. We will also need nonhospital data and other methods to verify representativeness of the hospital discharges.

We were able to produce three generalisable models that need testing, and this work will be carried out in the next stage of the research. Where nations can contribute to a minimum data set, valuable information can be assembled to inform health issues. However, even if it remains uncertain that the models give the right answer, the project has still revealed much more detail about valid international comparisons using routine hospital morbidity data. There is still much work to be done, and we look forward to presenting the results of this further work.

References

3M United Kingdom (Home page of 3M UK) (Online) Available : http://cms.3m.com/cms/GB/en/0-12/iilruFU/view.j h tml (Accessed September 2002).

Australian Institute of Health and Welfare (AIHW), The National Hospital Morbidity Database. (Home page AIHW) (Online) 7 August 2000 last updated. Available: http://www.aihw.gov.au/. (Accessed August 2002).

Centers for Disease Control and Prevention (Home page of CDC) (Online) 28 February 2003 last updated. Available: http://www.cdc.gov/scientific.htm (Accessed June 2002).

Centers for Medicare and Medicaid Services (Home page of CMS) (Online) May 2003 last updated. Available: http://www.cms.hhs.gov/ (Accessed: May 2002).

Centre for Epidemiology, Sweden (Home page National Board of Health and Welfare). (Online) 11 December 2001 last updated. Available: http://www.sos.se/sosmenye.htm. (Accessed 31 May 2002).

Danmarks Statistik (Statistics Denmark) (Home page of Danmarks Statisik) (Online) 31 March 2003 last updated. Available: http://www.dst.dk (Accessed June 2002).

De Lusignan S, Minmagh C, Kennedy J, Zeimet M, Bommezjin H, Bryant J. A survey to Idenitfy the Clinical Coding and Classification Systems Currently in use Across Europe. MEDINFO 2001; p.86-89.

Department of Health and Children, Ireland (Home page of DOH) (Online) 2003 last updated. Available: http://www.doh.ie/index.html (Accessed July 2002).

Department of Health, Social Services and Public Safety, Northern Ireland (Home page of DHSSPS) (Online) May 2003 last updated. Available: http://www.dhsspsni.gov.uk/ (Accessed August 2002).

HCA International Operations (Home page of HCA International) (Online) Available: http://www.stmartins-healthcare.co.uk/ (Accessed August 2002).

Health of Wales Information Service (Home page of HOWIS) (Online). Available: http://www.wales.nhs.uk/ (Accessed August 2002).

Health Solutions, Wales (Home page of Health Solutions Wales) (Online). Available: http://www.wales.nhs.uk/sites/home.cfm?orgid=166 (Accessed June 2002).

Healthcare Cost and Utilization Project (HCUP), USA (Home page of Agency for Healthcare research and policy) (Online) Available: http://www.ahrq.gov/data/hcup/ (Accessed: June 2002).

HIPE and NPRS Unit, ESRI (2002). Activity in Acute Public Hospitals in Ireland 1990-1999. Ireland: Department of Health and Children, ESRI.

Hospital Discharge Data Systems, DHHS (Home page of United States Department of Health and Human Services) (Online) Available: http://aspe.hhs.gov/statereg/HDDS.HTM (Accessed May 2002).

ICD-10-AM: International Statistical Classification of Diseases, 10th Revision, Australian Modification, First Edition (effective from July 1998). National Centre for Classification of Health.

ICD-9-CM - Australian Version of International Classification of Diseases, 9th Revision, Clinical Modification, Second Edition (effective from July 1996). National Coding Centre, Faculty of Health Sciences, University of Sydney, Sydney, Australia.

Iezzoni LI, Ash AS, Shwartz M, Mackiernan YD. Differences in procedure use, in-hospital mortality and illness severity by gender for acute myocardial infarction patients: are answers affected by data source and severity measure? Medical Care , 35(2), 1997: 158-71.

Iezzoni LI. Using Administrative Data to Study Persons with Disabilities. The Milbank Quarterly, Vol 80(2), 2002:347-79.

Medicare, USA (Home page of Medicare) (Online) Available: http://www.medicare.gov/ (Accessed July 2002).

Moise P (2001). Using Hospital Administrative Databases for a Disease-Based Approach to Studying Health Care Systems. OECD Ageing Related Disease Study, Technical Report.

National Board of Health and Welfare, Sweden (Online) April 2003 last updated. Available: http://www.sos.se/SOSE/about.htm (Accessed May 2002).

National Board of Health Denmark (Home page of Ministry of Health) (Online). Available: http://www.sst.dk/   (Accessed May 2002).

National Center for Policy Analysis (Home page of NCPA) (Online) Available: http://www.ncpa.org/ (Accessed August 2002).

National Center for Health Statistics, CDC (Home page of CDC) (Online) May 2003 last updated. Available: http://www.cdc.gov/nchs/ (Accessed June 2002).

National Health Service (NHS) Information Authority-UK (Home page of Department of Health) (Online) May 2003 - last updated. Available: http://www.doh.gov.uk/ (Accessed August 2002).

New Zealand Health Information Service (Home page of NZHIS) (Online) Available: http://www.nzhis.govt.nz/ (Accessed August 2002).

NordDRG Users' Manual Version 2002-NC-B2 (Home page of NordDRG Manual System) (Online) 21 October 2001last updated. Available: http://norddrg.kuntaliitto.fi/manual_2002/. (Accessed August 2002).

Nordic Centre for Classifications in Health Care (2002) NOMESCO Classification of Surgical Procedures (Online). Available: http://www.pubcare.uu.se/nordwho/index_e.htm. (Accessed August 2002).

Northern Ireland Statistics and Research Agency (Home page of NISRA) (Online)16 April 2003 last updated. Available: http://www.nisra.gov.uk/ (Accessed August 2002).

Organisation for Economic Co-operation and Development (Home page of OECD) (Online) Available: http://www.oecd.org (Accessed July 2002).

Private Health Care UK (Home page of Private HealthCare UK) (Online) 2002 last updated. Available: http://www.privatehealth.co.uk/ (Accessed August 2002).

Scotland's Health on the Web (Home page of SHOW) (Online) May 2003 last updated. Available: http://www.show.scot.nhs.uk/ (Accessed July 2002).

The Economic and Social Research Institute, Ireland (Home page of The Economic and Social Research Institute) (Online) Available: http://www.esri.ie/ (Accessed May 2002).

The Nordic Medico-Statistical Committee (NOMESCO) (Home page of NOMESCO) (Online) Available: http://www.nom-nos.dk/NOMESCO.HTM (Accessed August 2002).

US Census Bureau, United States Department of Commerce (Home page of US Census Bureau) (Online) Available: http://www.census.gov/ (Accessed May 2002).

US Congress, Office of Technology Assessment (OTA). (1994) International Health Statistics: What the Numbers Mean for the US (Online) Available: http://www.wws.princeton.edu/~ota/disk1/1994/9418_n.html (Accessed May 2002).

WHO Collaborating Centre for the Classification of Diseases in the Nordic Countries (2000). Report on Nordic meeting on the NOMESCO Classification of External Causes of Injuries (NCECI) Tuesday 3 October 2000, 10.00-16.00 (Online). Available: http://www-hotel.uu.se/socmed/nordwho/nyheter/Minutes-NCECI-Oct00.htm (Accessed 21 August 2002).

WHO Statistical Information System (Home page of WHOSIS) (Online) no date. Available: http://www3.who.int/whosis/menu.cfm . (Accessed 20 August 2002).

Zelmer J, Virani S, Alvarez R (1999). "Recent Developments in Health Information: An International Perspective." Paper Commissioned by the National Committee for Vital Health Statistics for a Workshop on Developing the 21st Century Vision for Health Statistics.