Identifying Patients with Hypertension: A Case for Auditing Electronic Health Record Data

by Adam Baus, MA, MPH; Michael Hendryx, PhD; and Cecil Pollard, MA

Abstract

Problems in the structure, consistency, and completeness of electronic health record data are barriers to outcomes research, quality improvement, and practice redesign. This nonexperimental retrospective study examines the utility of importing de-identified electronic health record data into an external system to identify patients with and at risk for essential hypertension.

We find a statistically significant increase in cases based on combined use of diagnostic and free-text coding (mean = 1,256.1, 95% CI 1,232.3–1,279.7) compared to diagnostic coding alone (mean = 1,174.5, 95% CI 1,150.5–1,198.3). While it is not surprising that significantly more patients are identified when broadening search criteria, the implications are critical for quality of care, the movement toward the National Committee for Quality Assurance’s Patient-Centered Medical Home program, and meaningful use of electronic health records. Further, we find a statistically significant increase in potential cases based on the last two or more blood pressure readings greater than or equal to 140/90 mm Hg (mean = 1,353.9, 95% CI 1,329.9–1,377.9).

Keywords: electronic health record, data quality, outcomes research, quality care, hypertension, registry

Introduction

The benefits of electronic health records (EHRs) for primary care and the application of these systems to outcomes research and current efforts in practice redesign such as the National Committee for Quality Assurance’s Patient-Centered Medical Home program are often hampered by barriers to full integration of EHRs. Common barriers include lack of trust in EHRs to securely store medical records,^1–5 physicians’ views that EHRs interfere with clinical judgment;^6,7 lack of standards in data formatting and lack of interoperability;^8-16 the required time, training, and investment to become proficient in using the systems;^17–22 the absence of local leadership to champion the systems;^23-27 difficulties in organizational redesign to use the EHR;^28-34 and lack of readiness to implement EHRs successfully.^35-37 We sought to examine one problem—the structure, consistency, and completeness of EHR data—by importing de-identified EHR data into an external system for analysis of diagnostic information.

Background

EHRs have the potential to be valuable tools for health outcomes research in primary care and a critical component in practice redesign and prevention of chronic diseases such as hypertension through identification of at-risk patients.^38-45 While manual review of medical records is resource intensive, using diagnosis codes stored within EHRs permits searching in a more comprehensive and efficient manner.^46-47 However, problems in the structure, consistency, and completeness of EHR data and the use of free-text entries rather than discrete data fields create barriers to research, outcomes reporting, and quality improvement activities, particularly among smaller, rural practices.^48–65

Given the challenges created by free-text data entry into EHRs, the current study examines the ability to identify cases with essential hypertension by importing de-identified EHR data from 11 West Virginia primary care centers into an external system, in this case a public-domain patient registry. An advantage of the registry is that it is accessible at the practice level and requires no programming or statistical expertise to use. This study examines whether patients with a diagnosis of essential hypertension are missed if searching only by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic codes (401.0–401.9). ICD-9-CM coding is currently used in this particular EHR system. We test the hypothesis that there will be significantly fewer patients identified with hypertension based on ICD-9-CM diagnosis codes relative to use of diagnosis codes plus free-text coding of hypertension. Support of this hypothesis would document the benefits of auditing EHR data for completeness and consistency, inform quality improvement efforts in overcoming barriers to EHR data quality and reliability, and support the National Committee for Quality Assurance health information framework, which highlights the need for interventions designed to improve the management and application of EHR data for research and quality improvement. Improving the management and application of EHR data has gained increased attention as a vital component in the overall success of health information technology endeavors.^66-76 Secondarily, we hypothesize that significantly more cases will be identified through a third measure based on the guidelines for diagnosis of hypertension presented in the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (two or more most recent blood pressure readings greater than or equal to 140/90 mm Hg among those without any diagnosis of essential hypertension).⁷⁷ This third measure identifies those at risk for or undiagnosed with hypertension, and would help document the benefits of analyzing EHR data in an external format.

Methods

This research is a nonexperimental retrospective study of essential hypertension cases identified across 11 West Virginia primary care centers using the same EHR system. We used a previously developed tool to import data from the EHRs into the Chronic Disease Electronic Management System (CDEMS).⁷⁸ CDEMS is a Microsoft Access–based public-domain registry originally developed by the Washington State Department of Health. Moving the de-identified EHR data to an external system, in this case a registry, allows for data transparency in that key data within the EHR (i.e., patient diagnoses, demographics, vitals, laboratory results and services) can be queried for coding consistency and completeness. Table 1 lists all data elements imported from the EHR into the registry. This registry was chosen as the tool for analysis because it is accessible by each primary care center, allowing methods and tools for this research to be applied at the practice level for quality improvement efforts in data management, identification of at-risk patients, and quality-of-care improvement. De-identified data included in this analysis are for all active patients in the 11 sites as of December 31, 2010.

Table 1
Registry Data Fields Imported from the Electronic Health Record

Queries were built in the registry to search the de-identified EHR data to:

1. identify unduplicated patients with a diagnosis of essential hypertension based on ICD-9-CM codes (using the diagnosis and demographic portions of the data);
2. identify unduplicated patients with a diagnosis of essential hypertension based on free-text entries (using the diagnosis and demographic portions of the data); and
3. identify unduplicated patients whose last two or more blood pressure readings were greater than or equal to 140/90 mm Hg and who did not have a documented diagnosis of essential hypertension in either ICD-9-CM code or free-text format (using the diagnosis, demographic, and vital signs portions of the data).

Table 2 provides a listing of queries used to identify patients, a description of the functions of each query, and a list of the free-text diagnoses that were detected. Identification of patients with a diagnosis of essential hypertension by ICD-9-CM code was accomplished by limiting search criteria to codes 401.0–401.9. Identification of patients with a free-text diagnosis of essential hypertension required more search steps. Search of the diagnosis field called “Other” required use of the LIKE condition function in Microsoft Access, also known as wildcards, to locate all matching criteria. Wildcard expressions used in the search were “*Hyperten*”; “*HTN*”; “*401*”; and “*Hyper ten*”. Upon review of all free-text results, it was evident that the search returned results relating to other forms of hypertension that needed to be excluded in this study. Excluded from search criteria were the following: “*Pulm*”; “*Neph*”; “*Coat*”; “*Retino*”; “*Pre*”; “*Ocul*”; “*Occul*”; “*Portal*”; “*Gastro*”; “*Partum*”; “*Ortho*”; “*Oval*”; “*Border*”; “*Myopathy*”; “*Medic*”; “*Renal*”; “*Venous*”; “*FH*”; “*Family*”; “*Gest*”; “*Decea*”; “*Tight*”; “*Disea*”; “*Infar*”; “*Intracran*”; and “*Elevated*”. Means, standard deviations, and 95 percent confidence intervals were calculated for the number of cases to measure the differences between each method.

Table 2
Query Names and Descriptions

Results

Based on use of ICD-9-CM codes alone, 12,919 unduplicated patients with essential hypertension were identified in the 11 sites. Searching free-text diagnoses, 898 additional unduplicated patients were identified. Broadening the search criteria to patients whose last two or more blood pressure readings were consistently greater than or equal to 140/90 mm Hg identifies an additional 1,076 unduplicated patients not identified by ICD-9-CM codes or free-text entries (range = 297). Use of all three methods identified 14,893 cases. Table 3 presents these findings.

Table 3
Count of Patients with Essential Hypertension, by Search Criteria

Placing confidence intervals around the means of each patient count method, we find a statistically significant increase in total cases identified with essential hypertension based on combined use of ICD-9-CM coding and free text (mean = 1,256.1, 95% CI = 1,232.3–1,279.7) compared to ICD-9-CM coding alone (mean = 1,174.5, 95% CI = 1,150.5–1,198.3). Furthermore, we find a statistically significant increase in identification of potential cases based on cases in which the last two or more blood pressure readings were greater than or equal to 140/90 mm Hg (mean = 1,353.9, 95% CI = 1,329.9–1,377.9) compared to ICD-9-CM coding plus free-text search. Use of only ICD-9-CM codes missed 13.3 percent of cases as identified using all three methods. Table 4 and Figure 1 present these findings.

Table 4
Increase in Count of Patients with Essential Hypertension, by Search Criteria and Primary Care Center

Figure 1: Increase in Count of Patients with Hypertension, by Search Criteria

Figure shows statistically significant increases in identification of essential hypertension cases using three search criteria methods. Electronic health record data are from all active patients in each primary care center as of December 31, 2010.

Discussion

By auditing EHR data in an external system, this study finds significant limitation in the ability to identify patients with a diagnosis of essential hypertension due to the use of free-text diagnosis entries. This study allows for the identification of a problem in data quality and completeness, and is translational in nature in that the study methods and tools are accessible to each site to monitor documentation processes and make adjustments as needed without time-intensive chart reviews or special programming. Further, importing the EHR data into an external system allows for analysis of blood pressure results to identify patients either undiagnosed with or at risk for development of the condition.

While it may not be surprising that significantly more patients are identified when broadening the search criteria, the implications are critical for quality of care because the identification of patients by health condition is a fundamental step in the process of applying data for quality improvement and reporting. Furthermore, the ability to accurately report data at the population level is central to the Patient-Centered Medical Home program and to meaningful use criteria. The inability to capture all patients by health condition yields reports for Patient-Centered Medical Home and meaningful use purposes that are inaccurate. Likewise, EHRs offer the promise of better patient care through decision support tools such as those that suggest care guidelines and treatment based on a patient’s health condition. However, problems in data quality can result in lower levels of provider trust in the data and the therefore decreased application of EHR data to patient care. Auditing data within the EHR can help identify these problems and provide the opportunity to correct them, for example, through training on the use of EHRs and through development of practice policies and procedures aimed at eliminating free-text entry of diagnoses. While it may not be feasible to alter the structure or functions of the EHR, it is reasonable to expect that quality improvement efforts centered on training and practice policies will help overcome barriers to data quality.

The substantial variability between clinics that was detected enables identification of clinics that follow best practices (i.e., those with relatively low proportions of diagnoses of hypertension recorded by free text, such as clinics E and F in Table 4), from which other sites can learn and apply documentation practices and policies. Likewise, this analysis aids in identifying clinics at which data management support and follow-up training is warranted (e.g., clinics J and G in Table 4). While clinic-level variability is not addressed within this study, results from this research allow for follow-up research efforts to be designed and conducted with these sites.

Conclusions

Our study reported significant loss in the ability to identify essential hypertension cases due to use of free-text coding. However, the study methods and tools offer translational opportunities at the primary care level, enabling each participating site to use these methods and tools to improve their own office procedures, training, and policies surrounding data entry into EHRs. This study highlights the need for training in data quality and management, even on basic levels such as using EHR templates and discrete fields for data entry rather than free-text fields. Targeted training is advisable because various members of a care team, such as physicians, nurses, medical assistants, and front-office staff, contribute data to the EHR at various points in the care process. Continued monitoring of these sites using tools developed in this research will help determine the long-term benefits of increased attention to EHR data quality. It is reasonable to expect that efforts to improve data quality will bolster improved integration of these systems while also facilitating the use of EHRs for quality-of-care improvement and efforts in practice redesign.

This study points to the need for future research. First, only essential hypertension was studied. Additional health conditions, such as other forms of hypertension, comorbid cardiovascular health conditions, diabetes, or chronic kidney disease, need to be examined. Second, the de-identified data are from only one EHR system. Future research needs to account for data from other systems to see if these findings are replicated. Third, patients with consistently high blood pressure readings need clinical follow-up to determine whether or not they have hypertension. Lastly, additional analyses are needed to account for patient age criteria (i.e., 25–79 years of age) when identifying patients with hypertension. The intent of this research was to identify patients with essential hypertension regardless of age or demographic criteria, thereby permitting initial exploration of the ability to conduct a more rigorous level of analysis of EHR data through importation of data into an external system.

Acknowledgments

The authors would like to acknowledge the support of ongoing partnership with the West Virginia Bureau for Public Health, Office of Community Health Systems and Health Promotion.

Adam Baus, MA, MPH, is the senior program coordinator for the Office of Health Services Research in the West Virginia University Department of Community Medicine in Morgantown, WV.

Michael Hendryx, PhD, is the director of the West Virginia Rural Health Research Center in Morgantown, WV.

Cecil Pollard, MA, is the director of the Office of Health Services Research in the West Virginia University Department of Community Medicine in Morgantown, WV.

Notes

¹ Wagner, K. A., F. W. Lee, A. W. White, D. M. Ward, and S. M. Ornstein. “Impact of an Electronic Medical Record System on Community-based Primary Care Clinics.” Journal of the American Board of Family Practice 13, no. 5 (2000): 338–48.

² Kaushal, R., D. W. Bates, E. G. Poon, A. K. Jha, and D. Blumenthal. “Functional Gaps in Attaining a National Health Information Network.” Health Affairs 24, no. 5 (2005): 1281–89.

³ Burton, L. C., G. F. Anderson, and I. W. Kues. “Using Electronic Health Records to Help Coordinate Care.” Milbank Quarterly 82, no. 3 (2004): 457–81.

⁴ Miller, R. H., and I. Sim. “Physicians’ Use of Electronic Medical Records: Barriers and Solutions.” Health Affairs 23, no. 2 (2004): 116–26.

⁵ Kristianson, K., H. Ljunggren, and L. Gustafsson. “Data Extraction from a Semi-structured Electronic Medical Record System for Outpatients: A Model to Facilitate the Access and Use of Data for Quality Control and Research.” Health Informatics Journal 15, no. 4 (2009): 305–19.

⁶ Rousseau, N., E. McColl, J. Newton, J. Grimshaw, and M. Eccles. “Practice Based, Longitudinal, Qualitative Interview Study of Computerised Evidence Based Guidelines in Primary Care.” British Medical Journal 326 (2003): 314–22.

⁷ Hanna, K. E., S. M. Anderson, and S. D. Maddox. Think Research: Using Electronic Medical Records to Bridge Patient Care and Research. Washington, DC: Center for Accelerating Medical Solutions, 2005.

⁸ Burton, L. C., G. F. Anderson, and I. W. Kues. “Using Electronic Health Records to Help Coordinate Care.”

⁹ Miller, R. H., and I. Sim. “Physicians’ Use of Electronic Medical Records: Barriers and Solutions.”

¹⁰ Hanna, K. E., S. M. Anderson, and S. D. Maddox. Think Research: Using Electronic Medical Records to Bridge Patient Care and Research.

¹¹ Amatayakul, M. “EHR? Assess Readiness First.” Healthcare Financial Management 59, no. 5 (2005): 112–13.

¹² Bristol, N. “The Muddle of US Electronic Medical Records.” Lancet 365, no. 9471 (2005): 1610–11.

¹³ Taylor, R., A. Bower, F. Girosl, J. Bigelow, K. Fonkych, and R. Hillstead. “Promoting Health Information Technology: Is There a Case for More-Aggressive Government Action?” Health Affairs 24, no. 5 (2005): 1234–45.

¹⁴ Goldschmidt, P. G. “HIT and MIS: Implications of Health Information Technology and Medical Information Systems.” Communications of the ACM 48, no. 10 (2005): 69–74.

¹⁵ Bradley, C. J., L. Penberthy, K. J. Devers, and D. J. Holden. “Health Services Research and Data Linkages: Issues, Methods, and Directions for the Future.” Health Services Research 45, no. 5 (2010): 1468–88.

¹⁶ Baron, R. J. “Quality Improvement with an Electronic Health Record: Achievable, But Not Automatic.” Annals of Internal Medicine 147, no. 8 (2007): 549–52.

¹⁷ Wagner, K. A., F. W. Lee, A. W. White, D. M. Ward, and S. M. Ornstein. “Impact of an Electronic Medical Record System on Community-based Primary Care Clinics.”

¹⁸ Kristianson, K., H. Ljunggren, and L. Gustafsson. “Data Extraction from a Semi-structured Electronic Medical Record System for Outpatients: A Model to Facilitate the Access and Use of Data for Quality Control and Research.”

¹⁹ Hersh, W. R. “Medical Informatics: Improving Health Care through Information.” Journal of the American Medical Association 288, no. 16 (2002): 1955–58.

²⁰ Walsh, S. H. “The Clinician’s Perspective on Electronic Health Records and How They Can Affect Patient Care.” British Medical Journal 328, no. 7449 (2004): 1184.

²¹ May, J. “The Electronic Medical Record: A Valuable Partner.” Journal of Medical Practice Management 21, no. 2 (2005): 100–102.

²² D. W. “Physicians and Ambulatory Electronic Health Records.” Health Affairs 24, no. 5 (2005): 1180–89.

²³ Wagner, K. A., F. W. Lee, A. W. White, D. M. Ward, and S. M. Ornstein. “Impact of an Electronic Medical Record System on Community-based Primary Care Clinics.”

²⁴ Burton, L. C., G. F. Anderson, and I. W. Kues. “Using Electronic Health Records to Help Coordinate Care.”

²⁵ Lorenzi, N. M., R. T. Riley, A. J. C. Blyth, G. Southon, and B. J. Dixon. “Antecedents of the People and Organizational Aspects of Medical Informatics.” Journal of the American Medical Informatics Association 4, no. 2 (1997): 79–93.

²⁶ Doolan, D. F., D. W. Bates, and B. C. James. “The Use of Computers for Clinical Care: A Case Series of Advanced US Sites.” Journal of the American Medical Informatics Association 10 (2003): 94–107.

²⁷ Miller, R. H., I. Sim, and J. Newman. Electronic Medical Records: Lessons from Small Physician Practices. San Francisco: California HealthCare Foundation, 2003.

²⁸ Baron, R. J. “Quality Improvement with an Electronic Health Record: Achievable, But Not Automatic.”

²⁹ Hersh, W. R. “Medical Informatics: Improving Health Care through Information.”

³⁰ Lorenzi, N. M., R. T. Riley, A. J. C. Blyth, G. Southon, and B. J. Dixon. “Antecedents of the People and Organizational Aspects of Medical Informatics.”

³¹ Bergh, M. “Patient Care Information Systems and Health Care Work: A Sociotechnical Approach.” International Journal of Medical Informatics 55, no. 2 (1999): 87–101.

³² Lorenzi, N. M., and R. T. Riley. “Managing Change: An Overview.” Journal of the American Medical Informatics Association 7, no. 2 (2000): 116–24.

³³ Kuhn, K. A., and D. A. Guise. “From Hospital Information Systems to Health Information Systems.” Methods of Medical Informatics 40, no. 4 (2001): 275–87.

³⁴ Vishwanath, A., S. R. Singh, and P. Winkelstein. “The Impact of Electronic Medical Record Systems on Outpatient Workflows: A Longitudinal Evaluation of Its Workflow Effects.” International Journal of Medical Informatics 79 (2010): 778–91.

³⁵ Burton, L. C., G. F. Anderson, and I. W. Kues. “Using Electronic Health Records to Help Coordinate Care.”

³⁶ Amatayakul, M. “EHR? Assess Readiness First.”

³⁷ Harris, K. D., J. T. Parsons, and R. Jimenez. “Developing a Methodology for the Customization of an Electronic Medical Record in a Rural Health Care Setting.” Arizona School of Health Sciences, Sun Life Community Health Center, n.d.

³⁸ Hanna, K. E., S. M. Anderson, and S. D. Maddox. Think Research: Using Electronic Medical Records to Bridge Patient Care and Research.

³⁹ Terry, A. L., V. Chevendra, A. Think, M. Stewart, J. N. Marshall, and S. Cejic. “Using Your Electronic Medical Record for Research: A Primer for Avoiding Pitfalls.” Family Practice 27 (2010): 121–26.

⁴⁰ Etheredge, L. M. “Creating a High-Performance System for Comparative Effectiveness Research.” Health Affairs 29, no. 10 (2010): 1761–67.

⁴¹ Dean, B. B., J. Lam, J. L. Natoli, Q. Butler, D. Aguilar, and R. J. Nordyke. “Use of Electronic Medical Records for Health Outcomes Research.” Medical Care Research and Review 66, no. 6 (2009): 611–38.

⁴² Weiner, M. G., J. A. Lyman, S. Murphy, and M. Weiner. “Electronic Health Records: High-Quality Electronic Data for Higher-Quality Clinical Research.” Informatics in Primary Care 15 (2007): 121–27.

⁴³ Persell, S. D., A. N. Kho, J. A. Thompson, and D. W. Baker. “Improving Hypertension Quality Measurement Using Electronic Health Records.” Medical Care 47, no. 4 (2009): 388–94.

⁴⁴ Farley, T. A., M. A. Dalal, F. Mostashari, and T. R. Frieden. “Deaths Preventable in the US by Improvements in the Use of Clinical Preventive Services.” American Journal of Preventive Medicine 38, no. 6 (2010): 600–609.

⁴⁵ US Preventive Services Task Force. “Screening for High Blood Pressure in Adults.” December 2007. Available at http://www.uspreventiveservicestaskforce.org/uspstf/uspshype.htm.

⁴⁶ Pakhomov, S., S. Bjornsen, P. Hanson, and S. Smith. “Quality Performance Measurement Using the Text of Electronic Medical Records.” Medical Decision Making 28, no. 4 (2008): 462–70.

⁴⁷ Persell, S. D., A. N. Kho, J. A. Thompson, and D. W. Baker. “Improving Hypertension Quality Measurement Using Electronic Health Records.”

⁴⁸ Hanna, K. E., S. M. Anderson, and S. D. Maddox. Think Research: Using Electronic Medical Records to Bridge Patient Care and Research.

⁴⁹ Terry, A. L., V. Chevendra, A. Think, M. Stewart, J. N. Marshall, and S. Cejic. “Using Your Electronic Medical Record for Research: A Primer for Avoiding Pitfalls.”

⁵⁰ Dean, B. B., J. Lam, J. L. Natoli, Q. Butler, D. Aguilar, and R. J. Nordyke. “Use of Electronic Medical Records for Health Outcomes Research.”

⁵¹ Weiner, M. G., J. A. Lyman, S. Murphy, and M. Weiner. “Electronic Health Records: High-Quality Electronic Data for Higher-Quality Clinical Research.”

⁵² Millery, M., and R. Kukafka. “Health Information Technology and Quality of Health Care: Strategies for Reducing Disparities in Underresourced Settings.” Medical Care Research and Review 67, no. 5 (2010): 268S-98S.

⁵³ Hoff, G., P. M. Ottestad, S. R. Skaflotten, M. Bretthauer, and V. Moritz. “Quality Assurance as an Integrated Part of the Electronic Medical Record: A Prototype Applied for Colonoscopy.” Scandinavian Journal of Gastroenterology 44 (2009): 1259–65.

⁵⁴ Nahm, M. L., C. F. Pieper, and M. M. Cunningham. “Quantifying Data Quality for Clinical Trials Using Electronic Data Capture.” PLoS One 3, no. 8 (2008): e3049.

⁵⁵ Forster, M., C. Bailey, M. Brinkhof, C. Graber, A. Boulle, M. Sophr, E. Balestre, et al. “Electronic Medical Record Systems, Data Quality and Loss to Follow-up: Survey of Antiretroviral Therapy Programmes in Resource-Limited Settings.” Bulletin of the World Health Organization 86 (2008): 939–47.

⁵⁶ Benin, A. L., G. Vitkauskas, E. Thornquist, E. D. Shapiro, J. Concato, M. Aslan, and H. M. Krumholz. “Validity of Using an Electronic Medical Record for Assessing Quality of Care in an Outpatient Setting.” Medical Care 43, no. 7 (2005): 691–98.

⁵⁷ Wrightson, W. “A Comparison of Electronic and Handwritten Anaesthetic Records for Completness of Information.” Anaesthesia and Intensive Care 38, no. 6 (2010): 1052–58.

⁵⁸ Chan, K. S., J. B. Fowles, and J. P. Weiner. “Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature.” Medical Care Research and Review 2010, no. 67 (2009): 503–27.

⁵⁹ Hanna, K. E., S. M. Anderson, and S. D. Maddox. Think Research: Using Electronic Medical Records to Bridge Patient Care and Research.

⁶⁰ Bristol, N. “The Muddle of US Electronic Medical Records.”

⁶¹ Taylor, R., A. Bower, F. Girosl, J. Bigelow, K. Fonkych, and R. Hillstead. “Promoting Health Information Technology: Is There a Case for More-Aggressive Government Action?”

⁶² Goldschmidt, P. G. “HIT and MIS: Implications of Health Information Technology and Medical Information Systems.”

⁶³ Harris, K. D., J. T. Parsons, and R. Jimenez. “Developing a Methodology for the Customization of an Electronic Medical Record in a Rural Health Care Setting.”

⁶⁴ Dean, B. B., J. Lam, J. L. Natoli, Q. Butler, D. Aguilar, and R. J. Nordyke. “Use of Electronic Medical Records for Health Outcomes Research.”

⁶⁵ Lee, J., C. Cain, S. Young, N. Chockley, and H. Burstin. “The Adoption Gap: Health Information Technology in Small Physician Practices.” Health Affairs 24, no. 5 (2005): 1364–66.

⁶⁶ Kristianson, K., H. Ljunggren, and L. Gustafsson. “Data Extraction from a Semi-structured Electronic Medical Record System for Outpatients: A Model to Facilitate the Access and Use of Data for Quality Control and Research.”

⁶⁷ Terry, A. L., V. Chevendra, A. Think, M. Stewart, J. N. Marshall, and S. Cejic. “Using Your Electronic Medical Record for Research: A Primer for Avoiding Pitfalls.”

⁶⁸ Dean, B. B., J. Lam, J. L. Natoli, Q. Butler, D. Aguilar, and R. J. Nordyke. “Use of Electronic Medical Records for Health Outcomes Research.”

⁶⁹ Farley, T. A., M. A. Dalal, F. Mostashari, and T. R. Frieden. “Deaths Preventable in the US by Improvements in the Use of Clinical Preventive Services.”

⁷⁰ Hoff, G., P. M. Ottestad, S. R. Skaflotten, M. Bretthauer, and V. Moritz. “Quality Assurance as an Integrated Part of the Electronic Medical Record: A Prototype Applied for Colonoscopy.”

⁷¹ Benin, A. L., G. Vitkauskas, E. Thornquist, E. D. Shapiro, J. Concato, M. Aslan, and H. M. Krumholz. “Validity of Using an Electronic Medical Record for Assessing Quality of Care in an Outpatient Setting.”

⁷² Chan, K. S., J. B. Fowles, and J. P. Weiner. “Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature.”

⁷³ Schneider, E. C., V. Riehl, S. Courte-Wienecke, D. M. Eddy, and C. Sennett. “Enhancing Performance Measurement: NCQA’s Road Map for a Health Information Network.” Journal of the American Medical Association 282, no. 12 (1999): 1184–90.

⁷⁴ Pandza, H. “Minimal Data Sets for Electronic Medical Records in Hospital Settings.” Acta Informatica Medica 17, no. 2 (2009): 71–73.

⁷⁵ Romano, M. J., and R. S. Stafford. “Electronic Health Records and Clinical Decision Support Systems.” Archives of Internal Medicine 171, no. 10 (2011): online version.

⁷⁶ Damberg, C. L., S. M. Shortell, K. Raube, R. R. Gillies, D. Rittenhouse, R. K. McCurdy, L. P. Casalino, and J. Adams. “Relationship between Quality Improvement Processes and Clinical Performance.” American Journal of Managed Care 16, no. 8 (2010): 601–6.

⁷⁷ National Heart, Lung, and Blood Institute. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (NIH Publication No. 04-523). National Heart, Lung, and Blood Institute, 2003.

⁷⁸ Chronic Disease Electronic Management System (CDEMS). “The CDEMS User Network.” Available at http://www.cdems.com.