Effectiveness of Hospital Emergency Department Regionalization and Categorization Policy on Appropriate Patient Emergency Care Use - A nationwide long-term observational study in Taiwan

DOI: https://doi.org/10.21203/rs.3.rs-36852/v1

Abstract

Background: The goal of this study was to examine the effect of hospital emergency department (ED) regionalization policy and the categorization of hospital emergency care policy (categorization policy) on patient appropriate ED use.

Methods: We conducted an observational study of the effect of emergency care policy intervention on patients' visits. Seven years of data from the Taiwan National Health Insurance Research Database (NHIRD) were examined. Taiwan implemented a nationwide three-tiered hospital ED regionalization and categorization policies in 2007 and 2009, respectively. There were 1,835,860 ED visits found among one million random samples from the NHIRD between 2005 and 2011. ED visits were categorized by the modified New York University ED algorithm. A time series analysis was performed to examine the change in the rate of appropriate ED use after the policy took effect.

Results: Total ED visits increased by 10.7% from 2005 to 2011. Average appropriate ED visit rate was 66.9% during the policy intervention. The trend in the appropriate ED visit rate showed no significant policy effect.

Conclusions: Provider side of regionalization and categorization policies did increase emergency care accessibility. However, regionalization and categorization policies no significant effect on patient appropriate ED use.

Background

Emergency department (ED) overcrowding has become a health services research issue worldwide [15]. Evidence shows that ED overcrowding may jeopardize patient safety[611]. According to the Asplin input-throughput-output model, the management of patient flow might drive ED overcrowding research [12]. The Asplin model proposed that ED crowding may be caused by 1. the input dimension, represented by inappropriate ED use by patients and 2. the provider-side factor of inadequate accessibility and emergency physician capability in emergency care [12]. However, the relationship among accessibility, capability and appropriateness of ED use remains unknown.

The National Academy of Sciences National Research Council published the innovative concept of matching critically ill patients with the appropriate health care facility [13]. The American Medical Association (AMA) published guidelines for the categorization of hospital emergency capability in 1973 [14]. The American College of Emergency Physicians (ACEP) recommended the use of ED categorization and suggested integrated networks of emergency care to match patient needs [15]. Kocher et al. proposed definitions and a conceptual framework for the categorization, designation, and regionalization of emergency care policy evaluation criteria [16]. These theoretical concepts have a significant influence on developing emergency care systems worldwide. Emergency care categorization-[1725] and regionalization-related [2633] empirical data have proven the effectiveness of emergency care. However, most studies focus on specific time-sensitive disease entities, specific age groups or a regional scale. To the best of our knowledge, no centralized data collection system exists that characterizes how ED capabilities interact with patient health-seeking behavior.

Hospital-based emergency care development in Taiwan began in the late 1980s [34]. After 30 years of efforts, Taiwan established the whole-island emergency care system, including prehospital emergency medical services (EMS), rescuer-responsive hospital designation, regionalization (regionalization policy) and the categorization of hospital emergency capabilities (CHEC) policy. Taiwan has a universal health care system that began in 1995. Registration with a family doctor is not compulsory, nor is there a strong family physician referral system; patients may visit the ED directly. Taiwan national health insurance offers loose explicit criterion for ED visits (Supplementary Table 1) without limitations. Taiwan's ED regionalization and categorization policies were implemented in 2007 and 2009, respectively. The primary objective of the regionalization was to increase the accessibility of emergency care. The categorization policy objectives included 1. increasing emergency care quality and 2. disclosing emergency care quality information to people and guiding them toward the appropriate use of emergency care. Data show that categorization, designation, and regionalization policies improve the transparency of emergency care services by disclosing quality information about the capability levels of prehospital emergency medical services, EDs and hospital emergency services, but the changes to health care-seeking behavior as a result of categorization information is unclear [16]. Taiwan's nationwide designation, regionalization and categorization of hospital emergency care systems can have an "ideal" effect on emergency care delivery systems through health policy interventions directed toward appropriate ED use. This study hypothesizes that the regionalization and categorization policies may increase user access and that emergency care quality and capability information disclosure may guide the people to choose an "appropriate" treatment setting, this improving the appropriate ED care use rate. In this paper, we use the nationwide sampled database to test the effect of the regionalization and categorization policies on appropriate ED use by patients.

Methods

Study design

This was a seven-year observational study of the effect of hospital emergency care policy interventions on appropriate ED use by patients. Our study included all ED visits between 1 January 2005 and 31 December 2011. We only left one event per visit date to avoid having the same record separated into more than one record (Fig. 1).

Setting

Timeline: regionalization and categorization policy

Taiwan has a population of 23 million, which is served by one universal health insurance scheme that provides comprehensive health care. The Taiwan Ministry of Health and Welfare (MoHW) adopted the guidelines of categorization, designation, and regionalization; we assessed the entire island's emergency care system in terms of these milestone policies. The emergency network plan started in 1989, and in the year 1995, the Emergency Medical Services Act marked the beginning of modern EMS in Taiwan [35], with a primary goal of establishing an infrastructure and decreasing regional disparity. The Taiwan MoHW established emergency care services through a sequence of regionalization and categorization policies. The regionalization policy mostly focuses on pre-hospital EMS networking and rescuer-responsive hospital ED designations. The categorization policy focuses on all emergency care capabilities and capacities at a hospital.

Regionalization policy

The Institute of Medicine (IoM) recommends that emergency care be regionalized to allow patients to get the "right care in the right place at the right time" in a timely and consistent manner [36]. Regionalization is the matching of medical resources to patient needs to maximize health benefits and outcomes while minimizing the cost and use of resources over a specified geographic area. The designation is an important component of regionalization [16]. In July 2007, Taiwan launched the Emergency Medical Services Act Addition and Amendment. The revised Emergency Medical Services Act has three significant changes: 1. introducing hospital-based ED categorization, which represents a substantial focus on ED medical capabilities for providing resuscitation; 2. establishing the medical director system to connect the hospital ED physician with the prehospital EMS; and 3. forming six regional emergency operations centers (REOCs). Our prehospital regionalization efforts, including MoHW onsite auditing with the categorization of hospital-based ED as a three-tiered approach to emergency care (severe, moderate and general), a small-scale trial of the trauma system and a stroke center designation. Moderate- and severe-tiered EDs undergo on-site audits by the MoHW, and general-tiered hospitals are designated by the local health authority. In rural and remote areas, local governments can designate a hospital-based ED as an emergency response hospital for regional prehospital triage, initial stabilization, and transfer. According to the Emergency Medical Services Act Amendment, the prehospital EMS is the responsibility of the fire department-based 119 system; it provides on-site prehospital triage and care and dispatches patients to the designated nearest appropriate hospital ED. The rescuer-responsive hospital ED must provide appropriate care for emergent patients in the region for which they are responsible, without any refusal; if the necessary service cannot be performed, they should provide assistance with patient referral, and inter-hospital transfer is mandated. The patient should be stabilized, the hospital ED must be in charge of inter-hospital transfer to other hospitals that can offer proper care, and the regional emergency operations center must be contacted for further assistance and coordination. The hierarchical system of emergent rescuer-responsive hospitals and designated emergency care hospitals must provide transport within 30 minutes for time-sensitive illnesses such as stroke, acute myocardial infarction, major trauma, high-risk pregnancies, and newborns. In 2007, the revised Emergency Medical Services Act was the landmark for reorganizing the network of emergency care systems in Taiwan. Six REOCs may dispatch EMS ambulances and rescue personnel from the local hospital to emergency patients and mass casualties. The REOC coordinates with the 119 EMS to send the patient to the nearest appropriate hospital. The average response times are 4.1–4.9 minutes in urban areas and up to 6.6 minutes in rural areas [35]. The prehospital EMS advanced life support's appropriate dispatch rate is approximately 37% [37]. The regionalization policy strategy first designated responsible hospitals in 61 subregions; then, every subregion was designated one hospital that would respond to acute illnesses and then organized into six medical regions. After every region has an adequate number of hospital EDs, the hospital EDs' ability is categorized. Every hospital under this policy provides proper ED services. In Taiwan, approximately 70% of emergency care patients are self-referred, and only approximately 30% is transferred by EMS [38]. The pre-hospital time (from call to arrival at the hospital) of the EMS system is 16.8 minutes on average and 26 minutes at the 90th percentile [38].

The categorization of hospital emergency capabilities policy

In 2009, the Taiwan MoHW promulgated the "Standards for the Classification of Hospital Emergency Medical Capabilities Policy." According to these standards, the MoHW horizontally categorized hospitals based on their emergency care capabilities and capacities (including the presence of a hospital ED, a high-impact time-sensitive diseases care team, and an ICU) into three tiers (severe, moderate and general). The primary focus of the categorization policy was the establishment of centers for trauma, cardiac catheterization, stroke, perinatal emergency care conditions, high-risk pregnancies, and pediatric intensive care. The Joint Commission of Twain accredits hospitals every four years, and severe-tiered hospitals are considered the last line for hospital referrals. At the end of 2009, there were 191 rescuer-responsive hospitals, including 24 severe-, 57 moderate-, and 109 general-tiered hospitals. This policy ensures the use of timely, continuous, effective, and collaborative multidisciplinary methods. This policy has two principal objectives: 1. To provide quality hospital emergency care, and 2. to provide the CHEC with information regarding whether people chose the appropriate treatment setting (right care in the right place), which can be used to provide people with the appropriate quality information to increase the appropriate use of the ED. Under the influence of designation, regionalization and categorization guidance, Taiwan achieved nationwide hospital-based ED designation, regionalization, and hospital emergency care categorization in July 2009. Therefore, the prehospital EMS, hospital ED, hospital emergency care teams and inter-hospital referral system are well established in Taiwan [39].

Estimated appropriate ED use

This study adopted the modified Billings/New York University ED (NYU-ED) algorithm [40], which considers probability thresholds of the emergent/ED care that is needed but preventable/avoidable (EDCNPA) and emergent/ED care that is needed and not preventable/avoidable (EDCNNPA) that are equal to or greater than 0.50 as emergent hospital-ED care needed (defined as appropriate ED use) [4144]. Moreover, the authors reclassified inappropriate and unclassifiable ED visits by using explicit procedure- and outcome-based criteria to reflect the complexity and intensity of appropriate ED use [4553]. Injury and behavioral health ICD-9 codes were excluded in the original paper [40].

Data access and cleaning methods

The analytic data was derived from the Taiwan National Health Insurance Research Database (NHIRD) and is routinely collected administrative data from the National Health Insurance (NHI) [54]. This database reimburses approximately 99.7% of national health care services; its data include hospital and medical professional staff characteristics and subject's enrolment and medical information. This database has been shown to reflect the accessibility of health care facilities and medical care utilization [55, 56]. The medical information collected includes diagnosis, procedure, medication, medical cost, and date of visit [57]. The accuracy of the diagnoses recorded by the NHIRD for time-sensitive diseases, such as acute ischaemic stroke [58, 59], acute myocardial infarction [59], asthma [60] and pneumonia [61], has been validated. Encrypted unique personal identification numbers can link all the data to allow for longitudinal follow-up. This study used the Longitudinal Health Insurance Database 2005 (LHID2005), which comprises one million randomly sampled individuals, representing approximately 4.3% of the entire population of Taiwan. The LHID2005 sampled participants who were alive in 2005 and collected all the medical records for these individuals from 1995 onward. This study cohort is confirmed to represent Taiwan's population [62]. The study identified ED visits using the ED visits case type code and the case revenue code. We used these data to count ED visits. We obtained visit-level data for all ED visits to all hospitals in Taiwan from 2005 through 2011.

Data sources/ statistical measurement

We compared the baseline characteristics of the two policy periods using χ² tests and t-tests accounting for NHIRD administrative data. Since our data on ED use and its covariates is time-varying and longitudinal, we performed a time series analysis to estimate multilevel changes in appropriate ED use rates pre- and post-policy intervention. Before examining the policy's effect on the appropriate ED visit rate, the Dickey-Fuller test was used to determine the baseline appropriate use rate. The difference in mean monthly probabilities after the hospital-ED regionalization and categorization was determined using a segmented autoregressive integrated moving average (ARIMA) model with an indicator variable for the post-regionalization (on and after July 11, 2007) and post-categorization (on and after July 11, 2009) periods. The advantage of a segmented ARIMA model is that it can examine at the effect of events (such as the policy intervention) while taking autocorrelation and time effect into account. We also evaluated the odds of appropriate ED visit rates by examining them pre- and post-regionalization and categorization policy. All analyses were conducted using SAS 9.4 and Stata MP 14.0 software. The statistical significance was determined using a 2-tailed significance level of 0.05. Our study was approved by Institutional Review Board of National Yang-Ming University (YM107035E) on May, 5 2018.

Sensitivity and bias analysis

According to the ED crowding systematic review, common causes of ED crowding input factors are nonurgent visits, frequent flyers, and influenza season [63]. We tested the effects of frequent ED users ( ≧ 4 visits per year) [64] (Supplementary Fig. 1) and influenza in the year 2009 (Supplementary Fig. 2) [65]. Minus the 2009 pandemic influenza and frequent ED user effects, the trend in appropriateness classification showed no substantial changes. A sensitivity analysis was conducted using the summation probabilities of EDCNNPA and EDCNPA p ≧ 0.75 as the threshold (Supplementary Fig. 3). Similar trends in ED visit classification were confirmed to indicate no significant change.

Results

Emergency care access

The number of ED visits from 2005 to 2011 increased by 10.7%, ED expanse increased by 29.4%, and the number of ED physicians increased by 35.1%. However, the total number of hospitals decreased by 8.1%, and the number of EDs decreased by 8.6% during the study period (Supplementary Table 2).

Participant characteristics

A total of 1,931,451 ED visits were identified between 2005 and 2011 from one million sample datasets. Thus, 1,866,261 events constituted the study sample (Fig. 1). Of these, 65,190 events on the same date and at the same hospital were excluded to avoid having one visit separated among more than one record. In 30,401 cases in which two visits occurred on the same date but at different hospitals, we considered both visits as the same emergency event. A total of 76,460 visits related to injury and behavioral ICD-9 diagnoses were excluded from the NYU-ED original algorithm for international comparison. The crude appropriate ED visit rate was 66.9% in this study cohort. Table 1 reports the numbers and percentages for the baseline characteristics among the ED visits by policy intervention for observable samples. The age before regionalization was 39.6 ± 26.2 years; after regionalization (before categorization), it was 44.8 ± 25.3 years, and after categorization it was 46.2 ± 24.5 years. The Charlson Comorbidity Index (CCI) before regionalization was 1.0 ± 1.9; after regionalization (before categorization), it was 1.2 ± 2.1, and after categorization it was 1.2 ± 2.1.

Table 1

The baseline characteristic among the emergency department visit patients by policy intervention

 

Regionalization Policy Intervention

 

Categorization Policy Intervention

 
 

Before Regionalization

(n = 475,991)

 

After Regionalization

(n = 340,870)

 

Before Categorization

(n = 340,870)

 

After Categorization

(n = 448,873)

 

n

(%)

 

n

(%)

P Value

n

(%)

 

n

(%)

P Value

Sex

                       

Female

240,804

(50.6)

 

173,685

(51.0)

0.034

173,685

(51.0)

 

228,441

(50.9)

0.72

Male

235,187

(49.4)

 

167,185

(49.0)

 

167,185

(49.0)

 

220,432

(49.1)

 

Age

                       

< 18

116,178

(24.4)

 

58,052

(17.0)

< 0.001

58,052

(17.0)

 

65,081

(14.5)

< 0.001

18–64

251,361

(52.8)

 

189,921

(55.7)

 

189,921

(55.7)

 

261,301

(58.2)

 

≥ 65

108,452

(22.8)

 

92,897

(27.3)

 

92,897

(27.3)

 

122,491

(27.3)

 

Charlson Comorbidity Index

                       

CCI ≤ 1

371,818

(78.1)

 

251,609

(73.8)

< 0.001

251,609

(73.8)

 

328,616

(73.2)

0.002

CCI > 1

104,173

(21.9)

 

89,261

(26.2)

 

89,261

(26.2)

 

120,257

(26.8)

 

Place of residence

                       

Urban

119,532

(25.1)

 

86,117

(25.3)

0.11

86,117

(25.3)

 

114,076

(25.4)

0.58

Suburban

147,321

(31.0)

 

105,668

(31.0)

 

105,668

(31.0)

 

138,697

(30.9)

 

Rural

202,969

(42.6)

 

145,196

(42.6)

 

145,196

(42.6)

 

190,850

(42.5)

 

Missing

6,169

(1.3)

 

3,889

(1.1)

 

3,889

(1.1)

 

5,250

(1.2)

 

Place of ED resources

                       

Adequate area

385,958

(81.1)

 

277,692

(81.5)

0.005

277,692

(81.5)

 

364,120

(81.1)

0.012

Deprivation area

90,033

(18.9)

 

63,178

(18.5)

 

63,178

(18.5)

 

84,753

(18.9)

 

Day of visit

                       

Weekday

306,106

(64.3)

 

220,022

(64.5)

0.030

220,022

(64.5)

 

291,980

(65.0)

< 0.001

Weekend

169,885

(35.7)

 

120,848

(35.5)

 

120,848

(35.5)

 

156,893

(35.0)

 

Income level

                       

Quintile 1 (Lowest)

163,027

(34.3)

 

34,632

(10.2)

< 0.001

34,632

(10.2)

 

45,746

(10.2)

< 0.001

Quintile 2

43,972

(9.2)

 

105,197

(30.9)

 

105,197

(30.9)

 

134,040

(29.9)

 

Quintile 3

124,805

(26.2)

 

85,505

(25.1)

 

85,505

(25.1)

 

37,760

(8.4)

 

Quintile 4

46,384

(9.7)

 

40,680

(11.9)

 

40,680

(11.9)

 

126,315

(28.1)

 

Quintile 5 (Highest)

95,135

(20.0)

 

70,695

(20.7)

 

70,695

(20.7)

 

98,837

(22.0)

 

Missing

2,668

(0.6)

 

4,161

(1.2)

 

4,161

(1.2)

 

6,175

(1.4)

 

Occupation

                       

Dependents of the insured individuals

205,782

(43.2)

 

128,140

(37.6)

< 0.001

128,140

(37.6)

 

156,843

(34.9)

< 0.001

Civil servants, teachers, military personnel, and veterans

22,276

(4.7)

 

24,607

(7.2)

 

24,607

(7.2)

 

31,548

(7.0)

 

Nonmanual workers and professionals

83,206

(17.5)

 

60,068

(17.6)

 

60,068

(17.6)

 

87,623

(19.5)

 

Manual workers

114,506

(24.1)

 

92,937

(27.3)

 

92,937

(27.3)

 

126,101

(28.1)

 

Other

47,737

(10.0)

 

31,090

(9.1)

 

31,090

(9.1)

 

40,696

(9.1)

 

Missing

2,484

(0.5)

 

4,028

(1.2)

 

4,028

(1.2)

 

6,062

(1.4)

 
Note: The policy of regionalization started on 13 July 2007. Moreover, the policy of categorization began on 13 July 2009. However, using the 31 July 2007 and 31 July 2009 as the cut of point.

Appropriate ED visit rate by baseline characteristics for different policy interventions

Table 2 presents the overall rate of appropriate ED visits for regionalization implementation (64.1% before vs. 68.1% post-regionalization policy); no significant change occurred after the categorization policy was implemented (68.6% before vs. 68.5% post-categorization policy). Patients who were male, aged ≥ 65 years, had a CCI > 1, lived in a rural region, lived in an area with inadequate emergent care resources, visited on a weekday, had an income level in quintile three, and were in occupational groups including civil servants, teachers, military personnel, and veterans had a higher rate of appropriate ED use.

Table 2

Appropriate rate by baseline characteristic on different policy intervention

 

Regionalization Policy Intervention, n = 816,861

   

Categorization Policy Intervention, n = 789,743

 
 

Before Regionalization

(n = 475,991)

 

After Regionalization

(n = 340,870)

P Value

 

Before Categorization

(n = 340,870)

 

After Categorization

(n = 448,873)

P Value

No of Appropriate ED Visit

(%)

 

No of Appropriate ED Visit

(%)

 

No of Appropriate ED Visit

(%)

 

No of Appropriate ED Visit

(%)

Total

305,116

(64.1)

 

233,970

(68.6)

< 0.001

 

233,970

(68.6)

 

307,261

(68.5)

0.17

Sex

                         

Female

153,088

(63.6)

 

117,849

(67.9)

< 0.001

 

117,849

(67.9)

 

154,642

(67.7)

0.38

Male

152,028

(64.6)

 

116,121

(69.5)

< 0.001

 

116,121

(69.5)

 

152,619

(69.2)

0.28

Age

                         

< 18

43,598

(37.5)

 

21,956

(37.8)

0.29

 

21,956

(37.8)

 

23,658

(36.4)

< 0.001

18–64

165,836

(66.0)

 

129,437

(68.2)

< 0.001

 

129,437

(68.2)

 

174,718

(66.9)

< 0.001

≥ 65

95,682

(88.2)

 

82,577

(88.9)

0.006

 

82,577

(88.9)

 

108,885

(88.9)

1.00

Charlson Comorbidity Index

                         

CCI ≤ 1

211,153

(56.8)

 

153,181

(60.9)

< 0.001

 

153,181

(60.9)

 

198,990

(60.6)

0.027

CCI > 1

93,963

(90.2)

 

80,789

(90.5)

0.21

 

80,789

(90.5)

 

108,271

(90.0)

0.078

Place of residence

                         

Urban

72,188

(60.4)

 

55,220

(64.1)

< 0.001

 

55,220

(64.1)

 

72,620

(63.7)

0.13

Suburban

92,955

(63.1)

 

71,748

(67.9)

< 0.001

 

71,748

(67.9)

 

93,557

(67.5)

0.058

Rural

136,783

(67.4)

 

104,686

(72.1)

< 0.001

 

104,686

(72.1)

 

138,021

(72.3)

0.25

Missing

3,190

(51.7)

 

2,316

(59.6)

< 0.001

 

2,316

(59.6)

 

3,063

(58.3)

0.32

Place of ED resources

                         

Adequate area

245,981

(63.7)

 

189,216

(68.1)

< 0.001

 

189,216

(68.1)

 

246,829

(67.8)

0.021

Deprivation area

59,135

(65.7)

 

44,754

(70.8)

< 0.001

 

44,754

(70.8)

 

60,432

(71.3)

0.12

Day of visit

                         

Weekday

207,817

(67.9)

 

159,223

(72.4)

< 0.001

 

159,223

(72.4)

 

210,381

(72.1)

0.042

Weekend

97,299

(57.3)

 

74,747

(61.9)

< 0.001

 

74,747

(61.9)

 

96,880

(61.7)

0.61

Income level

                         

Quintile 1 (Lowest)

105,959

(65.0)

 

20,879

(60.3)

< 0.001

 

20,879

(60.3)

 

27,629

(60.4)

0.79

Quintile 2

29,545

(67.2)

 

77,534

(73.7)

< 0.001

 

77,534

(73.7)

 

98,340

(73.4)

0.13

Quintile 3

85,307

(68.4)

 

62,461

(73.0)

< 0.001

 

62,461

(73.0)

 

25,495

(67.5)

< 0.001

Quintile 4

27,359

(59.0)

 

26,183

(64.4)

< 0.001

 

26,183

(64.4)

 

89,831

(71.1)

< 0.001

Quintile 5 (Highest)

55,478

(58.3)

 

44,571

(63.0)

< 0.001

 

44,571

(63.0)

 

62,849

(63.6)

0.060

Missing

1,468

(55.0)

 

2,342

(56.3)

0.36

 

2,342

(56.3)

 

3,117

(50.5)

< 0.001

Occupation

                         

Dependents of the insured individuals

113,133

(55.0)

 

78,377

(61.2)

< 0.001

 

78,377

(61.2)

 

97,312

(62.0)

0.001

Civil servants, teachers, military personnel, and veterans

17,384

(78.0)

 

19,879

(80.8)

< 0.001

 

19,879

(80.8)

 

25,137

(79.7)

0.012

Nonmanual workers and professionals

49,789

(59.8)

 

37,726

(62.8)

< 0.001

 

37,726

(62.8)

 

53,941

(61.6)

< 0.001

Manual workers

88,197

(77.0)

 

72,606

(78.1)

< 0.001

 

72,606

(78.1)

 

97,322

(77.2)

< 0.001

Other

35,280

(73.9)

 

23,154

(74.5)

0.22

 

23,154

(74.5)

 

30,531

(75.0)

0.20

Missing

1,333

(53.7)

 

2,228

(55.3)

0.24

 

2,228

(55.3)

 

3,018

(49.8)

< 0.001

Using the chi-squared test to check the percentage of an emergent visit from all emergency department visit by subgroup; ED: emergency department

Time series analysis of the policy intervention effect

Table 3

The segmented autoregressive integrated moving average model analysis of the percentage of ED visit

 

Regionalization Policy Intervention

 

Categorization Policy Intervention

 

β

SE

P Value

 

β

SE

P Value

Baseline trend

0.06

0.05

0.25

 

-0.08

0.06

0.16

Level change after regionalization

-0.79

0.74

0.28

 

-0.08

1.09

0.95

Trend change after regionalization

-0.07

0.06

0.23

 

0.01

0.06

0.92

Percentage of female

0.66

0.33

0.05

 

0.27

0.30

0.37

Mean Charlson score

30.83

2.42

< 0.001

 

33.88

3.93

< 0.001

Percentage of residence in urban

0.01

0.41

0.98

 

0.17

0.42

0.68

Percentage of deprivation area

0.26

0.39

0.51

 

0.18

0.52

0.73

Percentage of weekend

0.04

0.05

0.36

 

0.06

0.07

0.40

Percentage of income level at quintile 1 (Lowest)

0.08

0.04

0.04

 

-0.19

0.95

0.84

Percentage of the dependents of the insured individuals

-0.13

0.14

0.35

 

-0.29

0.12

0.02

AR1

0.05

0.17

0.77

 

-0.04

0.19

0.82

ED: emergency department

Discussion

In this seven-year policy intervention observational study, the provider side-related ED regionalization and categorization policy had no significant medium-term effect (two years) on the patients' patterns of appropriate ED use and only improved the EDs' realized accessibility. ED utilization increased by 10.7%, and ED expanse increased by 29.4%, during the seven-year observation period. The effectiveness of hospital ED regionalization and categorization in Taiwan has been documented in terms of improved patient outcomes for acute myocardial infarction [66], ischaemic strokes [33], and burn injury [67, 68]. In 2015, nearly 500 injuries from severe burns occurred following a color-dust explosion in Taiwan. The overall regionalization and categorization emergency care performance resulted in a low (2.4%) mortality rate for this mass casualty burn incident, compared with the 26.8% predicted by international statistics [69].

Aday and Andersen noted that health policy might improve access, thus increasing realized utilization [70]. Contemporary health policy is driven by an emphasis on appropriate health services utilization, such as avoiding overuse, misuse or underuse [71]. Previous studies concluded that ED utilization varies by insurance status, socioeconomic status, race, and other sociodemographic factors [19, 72]. Common factors for inappropriate ED use include greater trust in the hospital than in primary care [48] or greater convenience [38] and time savings [73, 74]. Previous Billings/NYU-ED classification algorithm [40, 75] designed to monitor different groups' ambulatory care sensitive conditions [76] or safety-net role of ED use [40, 77]. One Taiwanese national validation study disclosed that an increase in the availability of ambulatory care physicians or facilities that did not decrease non-emergency treatment in ED use [78].

From the user's view, off-office hours visits, including evening and weekend visits, accounted for 76.2% of ED visits. People tend to seek immediate or ED services for time-sensitive illnesses [79, 80]. Our study determined that provider-side ED policy implementation and quality information disclosure did not increase patients' appropriate use of the ED. The following are possible explanations for this absence of an effect: 1. It is difficult for prudent laypersons to judge whether their condition is urgent or nonurgent and where they should go for treatment. 2. The hospital emergency capabilities categorization ensures the comprehensive availability of laboratory services, image studies, and treatment 24 hours a day/365 days a year. The unintended consequence of hospital emergency care quality disclosure is that the hospital ED becomes the first choice for people seeking treatment for time-sensitive emergency conditions or convenience. The Taiwan MoHW proposed that a co-payment of $12 for an ED visit might reduce primary care-treatable ED visits. However, the NHI medical service payment standards dictate that the medical center ED co-payment is $15, the regional hospital ED co-pay is $10, and the local hospital ED co-pay $5. In comparison, the co-payment for outpatient clinical treatment at a medical center is $12; at a regional hospital, it is $8; at a local hospital, it is $2.7; and for a general practitioner visit, the co-payment is $1.7. Additionally, traffic and waiting time costs in the primary care setting must be considered, along with the minimum wage of $4.50 per hour [81]. Meanwhile, a nationwide emergency policy required the hospital emergency care system to become better able to meet patients' needs by improving accessibility through increased convenience (such as providing immediate access to an ED anytime) and availability (such as providing consultations with an available specialist within 30 minutes). Under such circumstances, the hospital ED offers greater time savings, convenience, and possibly greater cost-effectiveness for patients. These factors may explain the higher negative regionalization policy intervention effect on CCI Score ≤ 1 patients their ED appropriate use.

Two practical approaches for increasing the appropriateness of ED use include financial constraints and case management had been discussed [82]. A systematic review disclosed that financial measures may decrease ED visits but cannot increase appropriateness. However, the case management method can decrease ED visits and increase appropriateness [82]. Our data examined the effects of a provider side-related policy and the disclosure of related information in terms of guiding patients toward appropriate ED use and found that this policy goal did not work. We agree with the Smulowitz, Friedman [83] suggestion to reshape emergency care and extend services for medical emergencies to meet patients' needs, such as offering real-time "face-to-face" telehealth to provide guidance and medical recommendations that support patient decision making and relieve anxiety and implementing an access policy that combines primary care and ED care data without time or location limitations [84, 85].

Strengths and limitations

This seven-year natural observation of the effects of an ED policy intervention on patient behavior has several strengths. 1. We provide real-world empirical data to explain the relationship between health policy interventions and patients' health-seeking behaviors. 2. We ensured that the effects of hospital ED regionalization and categorization policies on appropriate ED use were not caused by other confounders. The authors analyzed other possible causes of input factors, such as the 2009 H1N1 influenza pandemic and the frequent ED user effect. 3. To the best of our knowledge, this study is the first nationwide single insurance system example that supports the AMA, ACEP and Kocher et al. theoretical models regarding the categorization, designation, and regionalization of emergency care; our example can provide unique information for academic research in emergency care. Limitations for generalizability include the following: 1. This was a retrospective study using a dataset collected for administrative claims purposes according to the conceptualization of appropriate ED visits, which may be defined at the patient level, disease level, hospital level, and social context level [46, 8691]. In this study, we did not have enough information to address these holistic concerns. 2. The NHIRD administrative dataset is collected annually for reimbursement purposes, and there is natural attrition due to aging, migration, and death. 3. Categorization and designation are essential components in the regionalization of emergency care networks [92]. However, our emergency care policy/services had an established regionalization and categorization sequence, and these paradigm differences deserve further investigation. 4. In the market-maximized approach, financial and managed care strategies are chosen to drive improvement in appropriate ED use [93]. However, Taiwan MoHW chose a market-minimized policy-driven to guide appropriate ED use, which may limit the external validity of our study.

Conclusion

The provider side changes implemented by the hospital-based ED regionalization and CHEC policies did improve patient emergency care accessibility. However, emergency care quality disclosure may not guide patients toward more appropriate ED use. Strategies for balancing patients' needs and appropriate ED use require further investigation.

Abbreviations

ACEP

American College of Emergency Physicians; ACF:autocorrelation function; AIC:Akaike information criterion; AMA:American Medical Association; AR:autoregressive processes; ARIMA:autoregressive integrated moving average; BIC:Bayesian information criterion; CCI:Charlson Comorbidity Index; CHEC:categorization of hospital emergency capabilities; ED:emergency department; EDCNNPA:ED care needed and not preventable/avoidable; EDCNPA:ED care needed preventable/avoidable; emergency medical services (EMS); ICU:intensive care unit; LHID2005:Longitudinal Health Insurance Database 2005; MA:moving average processes; MoHW:Ministry of Health and Welfare; NHI:National Health Insurance;

NHIRD

National Health Insurance Research Database; NYU-ED algorithm:New York University ED algorithm; OPD:outpatient department; PACF:partial autocorrelation function; REOC:regional emergency operations center; USD:US Dollar.

Declarations

Ethics approval and consent to participate

This study approved by Institutional Review Board of National Yang-Ming University-YM107035E on May, 5 2018. In accordance with regulations of the National Health Research Institutes, patient identification information was anonymized, such that informed consent was not required.

Consent for publication

Not applicable.

Availability of data and materials

The data that support the findings of this study are available from Taiwan National Health Insurance Research Database but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the academic request and with permission of Taiwan National Health Insurance Administration.

Competing interests

The authors declare that they have no competing interests.

Funding

The authors declare that they have no funding.

Authors' contributors

C-Y.L. study concept and design, analysis and interpretation of data, preparation of the manuscript. Y-C.L. supervision for the development of study concept and design, analysis and interpretation of data, preparation of the manuscript. All authors have read and approved the manuscript.

Acknowledgments

We acknowledge the Taiwan National Health Insurance Research Database, which was provided by the National Health Insurance Administration, and is managed by National Health Research Institutes.

Author information

1Department of Neurology, Taipei City Hospital, Taipei, Taiwan

2Institute of Health and Welfare Policy, School of Medicine, National Yang-Ming University, Taipei, Taiwan

3Department of Health Care Management, National Taipei University of Nursing and Health, Taipei, Taiwan

4Master Program on Trans-disciplinary Long-Term Care and Management, National Yang-Ming University, Taipei, Taiwan

References

  1. Velt KB, et al. Emergency department overcrowding: a survey among European neurotrauma centres. Emerg Med J. 2018;35(7):447–8.
  2. Boyle A, et al. Comparison of the International Crowding Measure in Emergency Departments (ICMED) and the National Emergency Department Overcrowding Score (NEDOCS) to measure emergency department crowding: pilot study. Emerg Med J. 2016;33(5):307–12.
  3. Wang Z, et al. Causes of Emergency Department Overcrowding and Blockage of Access to Critical Services in Beijing: A 2-Year Study. J Emerg Med. 2018;54(5):665–73.
  4. Horwitz LI, Bradley EH. Percentage of US emergency department patients seen within the recommended triage time: 1997 to 2006. Arch Intern Med. 2009;169(20):1857–65.
  5. Wallingford G Jr, et al. Introduction of a Horizontal and Vertical Split Flow Model of Emergency Department Patients as a Response to Overcrowding. J Emerg Nurs. 2018;44(4):345–52.
  6. Trzeciak S, Rivers EP. Emergency department overcrowding in the United States: an emerging threat to patient safety and public health. Emergency medicine journal. 2003;20(5):402–5.
  7. Pines JM, Hollander JE. Emergency department crowding is associated with poor care for patients with severe pain. Ann Emerg Med. 2008;51(1):1–5.
  8. Cowan RM, Trzeciak S. Clinical review: emergency department overcrowding and the potential impact on the critically ill. Critical care. 2004;9(3):291.
  9. Schull MJ, et al. Emergency department crowding and thrombolysis delays in acute myocardial infarction. Ann Emerg Med. 2004;44(6):577–85.
  10. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ. 2016;353:i2139.
  11. Momeni M, et al. Emergency Overcrowding Impact on the Quality of Care of Patients Presenting with Acute Stroke. Adv J Emerg Med. 2018;2(1):e3.
  12. Asplin BR, et al., A conceptual model of emergency department crowding. Annals of emergency medicine, 2003A. 42(2): p. 173–180.
  13. National Academy of Sciences (US). National Research Council (US) Committee on Trauma National Research Council (US) Committee on Shock, Accidental Death and Disability: The Neglected Disease of Modern Society. Washington (DC): National Academies Press (US);. 1966. Available from: https://www.ncbi.nlm.nih.gov/books/NBK222962/ doi: 10.17226/9978. 1966: National Academies.
  14. American Medical Association, Recommendations of the Conference on the Guidelines for the Categorization of Hospital Emergency Capabilities. 1971: American Medical Association.
  15. Mehrotra A, et al. Important historical efforts at emergency department categorization in the United States and implications for regionalization. Acad Emerg Med. 2010;17(12):e154-60.
  16. Kocher KE, et al. Categorization, designation, and regionalization of emergency care: definitions, a conceptual framework, and future challenges. Acad Emerg Med. 2010;17(12):1306–11.
  17. Boyd DR, et al. Categorization of hospital emergency medical capabilities in Illinois: A statewide experience. IMJ Ill Med J. 1974;146(1):33–8.
  18. Detmer DE, et al. Regional categorization and quality of care in major trauma. J Trauma. 1977;17(8):592–9.
  19. Gibson G. Categorization of hospital emergency capabilities: some empirical methods to evaluate appropriateness of emergency department utilization. J Trauma. 1978;18(2):94–102.
  20. Grumbach K, et al. Regionalization of cardiac surgery in the United States and Canada: geographic access, choice, and outcomes. Jama. 1995;274(16):1282–8.
  21. DuBose JJ, et al., Effect of trauma center designation on outcome in patients with severe traumatic brain injury. Arch Surg, 2008. 143(12): p. 1213-7; discussion 1217.
  22. Mechem CC, et al. Resuscitation center designation: recommendations for emergency medical services practices. Prehosp Emerg Care. 2010;14(1):51–61.
  23. Nguyen YL, Kahn JM, Angus DC. Reorganizing adult critical care delivery: the role of regionalization, telemedicine, and community outreach. Am J Respir Crit Care Med. 2010;181(11):1164–9.
  24. Hunt MM, et al. The utility of a "trauma 1 OP" activation at a level 1 pediatric trauma center. J Pediatr Surg. 2017;52(2):322–6.
  25. Windorski J, et al., Differences in hospital outcomes following traumatic injury for patients experiencing immediate transfer to a level I trauma facility versus resuscitation at a critical access hospital (CAH). Am J Surg, 2018.
  26. Lyle DM, et al. Regionalization of trauma services in western Sydney: predicted effect on ambulance and hospital utilization rates. Aust N Z J Surg. 1991;61(8):589–96.
  27. Sampalis JS, et al., Trauma care regionalization: a process-outcome evaluation. J Trauma, 1999. 46(4): p. 565 – 79; discussion 579 – 81.
  28. Miyata H, et al. Toward quality improvement of cardiovascular surgery in Japan: an estimation of regionalization effects from a nationwide survey. Health Policy. 2009;91(3):246–51.
  29. Diaz JJ Jr, et al. Does regionalization of acute care surgery decrease mortality? J Trauma. 2011;71(2):442–6.
  30. Glickman SW, et al. Assessment of temporal trends in mortality with implementation of a statewide ST-segment elevation myocardial infarction (STEMI) regionalization program. Ann Emerg Med. 2012;59(4):243–52.e1.
  31. Spaite DW, et al. Statewide regionalization of postarrest care for out-of-hospital cardiac arrest: association with survival and neurologic outcome. Ann Emerg Med. 2014;64(5):496–506.e1.
  32. Kelly ML, et al. Decreased mortality in traumatic brain injury following regionalization across hospital systems. J Trauma Acute Care Surg. 2015;78(4):715–20.
  33. Tung YC, Chang GM. The Relationships Among Regionalization, Processes, and Outcomes for Stroke Care: A Nationwide Population-based Study. Medicine. 2016;95(15):e3327.
  34. Bullard MJ, Liaw SJ, Chen JC. Emergency medicine development in Taiwan. Ann Emerg Med. 1996;28(5):542–8.
  35. Chiang W-C, et al. EMS in Taiwan: past, present, and future. Resuscitation. 2009;80(1):9–13.
  36. Navein J, McNeil I. The Surrey Emergency Care System: a countywide initiative for change. Emerg Med J. 2003;20(2):192–5.
  37. Lu T-C, et al. The demand for prehospital advanced life support and the appropriateness of dispatch in Taipei. Resuscitation. 2006;71(2):171–9.
  38. Tsai JC, Liang YW, Pearson WS. Utilization of emergency department in patients with non-urgent medical problems: patient preference and emergency department convenience. J Formos Med Assoc. 2010;109(7):533–42.
  39. Ministry of Health. The Statistics and Trends in Health and Welfare 2014. Ministry of Health and Welfare: Taiwan; 2015.
  40. Billings J, Parikh N, Mijanovich T, Emergency department use: the New York story [Internet]. New York (NY): Commonwealth Fund; 2000 Nov [cited 2017 April 15].(Issue Brief No. 434). 2000.
  41. Ballard DW, et al., Validation of an algorithm for categorizing the severity of hospital emergency department visits. Medical care, 2010. 48(1).
  42. Gandhi SO, Sabik L. Emergency department visit classification using the NYU algorithm. Am J Manag Care. 2014;20(4):315–20.
  43. Wharam JF, et al. Emergency department use and subsequent hospitalizations among members of a high-deductible health plan. Jama. 2007;297(10):1093–102.
  44. Ballard DW, et al. Validation of an algorithm for categorizing the severity of hospital emergency department visits. Med Care. 2010;48(1):58–63.
  45. Lowy A, Kohler B, Nicholl J. Attendance at accident and emergency departments: unnecessary or inappropriate? J Public Health. 1994;16(2):134–40.
  46. Jeffery MM, et al. Validation of an algorithm to determine the primary care treatability of emergency department visits. BMJ open. 2016;6(8):e011739.
  47. Pereira S, et al. Appropriateness of emergency department visits in a Portuguese university hospital. Ann Emerg Med. 2001;37(6):580–6.
  48. Sempere-Selva T, et al. Inappropriate use of an accident and emergency department: magnitude, associated factors, and reasons—an approach with explicit criteria. Ann Emerg Med. 2001;37(6):568–79.
  49. Wolinsky FD, et al., Emergency department utilization patterns among older adults. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences, 2008. 63(2): p. 204–209.
  50. Lowe RA, Fu R. Can the emergency department algorithm detect changes in access to care? Academic emergency medicine. 2008;15(6):506–16.
  51. Miller S. The effect of insurance on emergency room visits: an analysis of the 2006 Massachusetts health reform. J Public Econ. 2012;96(11–12):893–908.
  52. Parboosingh EJ, Larsen DE, FACTORS INFLUENCING FREQUENCY AND APPROPRIATENESS OF UTILIZATION OF THE EMERGENCY ROOM BY THE ELDERLY. Medical Care, 1987. 25(12): p. 1139–1147.
  53. Lin C-Y, Lee Y-C. Appropriateness of emergency care use: a retrospective observational study based on professional versus patients’ perspectives in Taiwan. BMJ open. 2020;10(5):e033833.
  54. National Health Research Institutes. National Health Insurance Research Database. Accessed August 20,. 2018. https://nhird.nhri.org.tw/en/. Accessed August 20, 2018.]; Available from: https://nhird.nhri.org.tw/en/.
  55. Yang YT, et al. The relationship between accessibility of healthcare facilities and medical care utilization among the middle-aged and elderly population in Taiwan. Int J Qual Health Care. 2015;27(3):222–31.
  56. Lam C, et al. Emergency department utilization can indicate early diagnosis of digestive tract cancers: A population-based study in Taiwan. Comput Methods Programs Biomed. 2014;115(3):103–9.
  57. Hsieh C-Y, et al. Taiwan’s National Health Insurance Research Database: past and future. Clinical epidemiology. 2019;11:349.
  58. Cheng CL, et al. Validation of the National Health Insurance Research Database with ischemic stroke cases in Taiwan. Pharmacoepidemiol Drug Saf. 2011;20(3):236–42.
  59. Cheng C-L, et al. Validation of acute myocardial infarction cases in the national health insurance research database in Taiwan. Journal of epidemiology. 2014;24(6):500–7.
  60. Su VY-F, et al., Use of ICS/LABA combinations or LAMA is associated with a lower risk of acute exacerbation in patients with coexistent COPD and asthma. The Journal of Allergy Clinical Immunology: In Practice, 2018. 6(6): 1927–35. e3.
  61. Su VY-F, et al. Sleep apnea and risk of pneumonia: a nationwide population-based study. Cmaj. 2014;186(6):415–21.
  62. Longitudinal Health Insurance Database 2005(LHID2005). Accessed August 20, 2018; Available from: https://nhird.nhri.org.tw/en/Data_Subsets.html.
  63. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med, 2008. 52(2): 126–36. e1.
  64. van Tiel S, et al. Systematic review of frequent users of emergency departments in non-US hospitals: state of the art. Eur J Emerg Med. 2015;22(5):306–15.
  65. Feldman J. The NYU Classification System for ED Visits: WSHA Technical Concerns. accessed on November, 2017, 2010. 14: p. 2010.
  66. Tzeng IS, et al. Impact of performance grading on annual numbers of acute myocardial infarction-associated emergency department visits in Taiwan: Results of segmented regression analysis. Medicine. 2016;95(42):e4937.
  67. Yeong EK, et al. Response of a local hospital to a burn disaster: Contributory factors leading to zero mortality outcomes. Burns. 2018;44(5):1083–90.
  68. Lin CH, et al., Challenges of Burn Mass Casualty Incidents in the Prehospital Setting: Lessons From the Formosa Fun Coast Park Color Party. Prehosp Emerg Care, 2018: p. 1–5.
  69. Yang CC, Shih CL. A Coordinated Emergency Response: A Color Dust Explosion at a 2015 Concert in Taiwan. Am J Public Health. 2016;106(9):1582–5.
  70. Aday LA, Andersen R. A framework for the study of access to medical care. Health services research. 1974;9(3):208.
  71. Payne SM. Identifying and managing inappropriate hospital utilization: a policy synthesis. Health Serv Res. 1987;22(5):709–69.
  72. Hoot NR, Aronsky D. Systematic review of emergency department crowding: Causes, effects, and solutions. Ann Emerg Med. 2008;52(2):126–36.
  73. Payne K, et al. An after hours gp clinic in regional Australia: appropriateness of presentations and impact on local emergency department presentations. BMC Fam Pract. 2017;18(1):86.
  74. Coster JE, et al. Why Do People Choose Emergency and Urgent Care Services? A Rapid Review Utilizing a Systematic Literature Search and Narrative Synthesis. Acad Emerg Med. 2017;24(9):1137–49.
  75. Johnston KJ, et al. A “Patch” to the NYU Emergency Department Visit Algorithm. Health services research. 2017;52(4):1264–76.
  76. Billings J, et al. Impact of socioeconomic status on hospital use in New York City. Health Aff. 1993;12(1):162–73.
  77. Gordon JA. Emergency care as safety net. Health Aff (Millwood). 2000;19(2):277.
  78. Chan C-L, et al. The association between the availability of ambulatory care and non-emergency treatment in emergency medicine departments: a comprehensive and nationwide validation. Health Policy. 2013;110(2–3):271–9.
  79. Gill JM, Riley AW. Nonurgent use of hospital emergency departments: urgency from the patient's perspective. J Fam Pract. 1996;42(5):491–7.
  80. Chou S-C, et al. Analysis of a Commercial Insurance Policy to Deny Coverage for Emergency Department Visits With Nonemergent Diagnoses. JAMA Network Open. 2018;1(6):e183731–1.
  81. Luque Ramos A, Hoffmann F, Spreckelsen O. Waiting times in primary care depending on insurance scheme in Germany. BMC Health Serv Res. 2018;18(1):191.
  82. Raven MC, et al. The Effectiveness of Emergency Department Visit Reduction Programs: A Systematic Review. Ann Emerg Med. 2016;68(4):467–83 e15.
  83. Smulowitz PB, Friedman AB, Pines JM, Appropriations for "appropriate" visits: Payment denials for emergency department care. American Journal of Emergency Medicine, 2018.
  84. Dale J, et al. Safety of telephone consultation for “non-serious” emergency ambulance service patients. Quality Safety in Health Care. 2004;13(5):363–73.
  85. Todd TN, Using Data to Inform Appropriateness in Saskatchewan's Health Care System: A Snapshot of Low Acuity Emergency Department Use. 2017, Faculty of Graduate Studies and Research, University of Regina.
  86. Hopkins A, Fitzpatrick R, Foster A. What do we mean by appropriate health care. Quality in Health Care. 1993;2(1):17–1.
  87. David M, et al. Emergency outpatient services in the city of Berlin: Factors for appropriate use and predictors for hospital admission. European Journal of Emergency Medicine. 2006;13(6):352–7.
  88. Moss C, et al. Patient experience in the emergency department: inconsistencies in the ethic and duty of care. J Clin Nurs. 2015;24(1–2):275–88.
  89. Liberati A, et al. A European project assessing the appropriateness of hospital utilization: background, objectives and preliminary results. Int J Qual Health Care. 1995;7(3):187–99.
  90. Lowe RA, Abbuhl SB. Appropriate standards for “appropriateness” research. Ann Emerg Med. 2001;37(6):629–32.
  91. Tsai JC, Chen WY, Liang YW. Nonemergent emergency department visits under the National Health Insurance in Taiwan. Health Policy. 2011;100(2–3):189–95.
  92. Combes J, Arespacochaga E. Appropriate use of medical resources. 2013. American Hospital Association, Physician Leadership Forum.
  93. Charles E, Begley DRL, Robert O, Morgan PJ, Rowan. Rajesh Balkrishnan Evaluating the Healthcare System: Effectiveness, Efficiency, and Equity, Fourth Edition. Chicago: Health Administration, Press; 2013.