Evaluating the impact of a health policy exposed to an entire nationwide population presents challenges due to the absence of a control group. This restriction only permits pre- and post-implementation comparisons and complicates understanding the potential interplay of various disease shifts under the influence of policy is a significant challenge. Nevertheless, our study aims to thoroughly analyze such a policy, investigating its differential impacts across various time-sensitive diseases. By carefully examining each disease's guidelines, we discern the unique effects of the policy on each one. This process allows us to identify the groups affected by the policy and those serving as relatively unaffected counterfactuals, thereby giving us insights into the policy's impacts and difference-in-differences comparison.
The overall impact of the CHEC policy
Our study utilized real-world data to illustrate that the enforcement of the CHEC policy has led to a substantial reduction in overall medical orders and major diagnostic indicators. Nevertheless, there has been a conspicuous increase in diagnostic fees; this could be potentially attributed to stringent time indicators, as expedited diagnoses are crucial. Consequently, there could be an augmentation in the charges associated with other differential diagnoses. Moreover, the implementation of the CHEC policy has also resulted in a decrease in mortality rates, ultimately contributing to a reduction in overall medical expenses. Thus, the effectiveness and efficiency of the CHEC policy underscore its dual ability to decrease costs while improving patient outcomes. However, a crucial question remains unanswered: Why has the CHEC policy been effective in significantly reducing medical costs and mortality rates, even when the overall major diagnosis indicator has declined and no substantial change has been observed in treatment quality indicators? The following sections will delve further into diseases' individual and interactive effects to provide an analysis.
Individual disease response to CHEC policy
Implementing the CHEC policy aims to improve the quality of the diagnosis and treatment process, enable rapid diagnosis and timely treatment, efficiently reduce unnecessary examinations, and streamline management, leading to lower diagnostic cost and medical orders. As hospitals improve their capabilities in emergency medical care, the need for patient transfers is anticipated to reduce. Through the provision of timely and appropriate treatment, the ultimate objectives of the policy are to lower mortality rates and curtail overall healthcare utilization.
When examining individual disease policy effects, in the case of AIS, we observed a decrease in diagnostic indicators, which might be attributable to an increase in patient upward transfer rates and possibly suggests that hospital physicians may be more inclined to transfer AIS patients to higher-level hospitals to increase their chances of receiving thrombolytic treatment 28. Therefore, it presents a significant increase in the primary treatment indicators. There has been a substantial increase in diagnostic fees. c seemed to be adversely affected by the policy intervention, with increased medical orders, which led to increased diagnostic costs, upper transfer rates, long-term mortality rates, and overall medical costs. In the context of STEMI, there is an increase in diagnostic fees and a tendency to increase diagnostic and treatment indicators while transfer rates have decreased. This suggests a more intensive treatment approach for STEMI, correlating with decreased mortality rates. Conversely, there was a decrease in major diagnostic and treatment indicators and medical orders for septic shock and major trauma exhibited a decline in primary diagnosis indicators and primary treatment indicators also saw an increase in transfer rates. The differential impact on diagnostic costs, treatment indicators, transfer rates, long-term mortality rates, and medical costs among different diseases highlights the differential influence of policy effects. This could indirectly suggest the impact of the policy spotlight effect on AIS and STEMI conditions, while septic shock and major trauma may deteriorate due to the absence of policy monitoring.
Association of diseases with policy spotlight effect
Given the setting of this study, all emergency medical services are exposed to the CHEC policy, potentially subjecting emergency medical personnel to effects ranging from the Hawthorne effect and the policy spotlight effect. To differentiate the behaviors of healthcare providers under the influence of the Hawthorne effect or the policy spotlight effect, we selected AIS and STEMI diseases with well-established guidelines and time-based quality indicators under the CHEC policy. Septic shock, a disease with well-established guidelines but without specific time-based quality indicators, and major trauma, a disease without well-established guidelines or specific time-based quality indicators, were selected as external controls. Based on these natural quasi-experimental conditions, we hypothesize that the response observed may suggest varying levels of awareness among emergency care providers 29. The Hawthorne effect demonstrates that the productivity of individuals in an experiment may increase simply because they are being observed, reflecting the influence of human attention and intervention on behavior 30. The policy spotlight effect may be intensified by factors such as time constraints, ambiguous symptom patterns, and time-based quality surveillance indicators, prompting emergency care providers to unconsciously adopt selective behaviors concentrating on specific diseases according to policy targets 30. Consequently, diseases not prioritized by the policy, such as septic shock and major trauma in this study, may experience a significant decrease in process quality and outcome.
When using major trauma as a reference group, both AIS and STEMI, diseases monitored by time-based indicators, exhibited an increase in the medical orders corresponding to treatment indicators. Moreover, there was an apparent increase in diagnostic and overall healthcare expenses in the STEMI cohort and a decreased transfer rate. In contrast, no significant changes were observed for the septic shock group, which was not under time indicator monitoring when compared to the major trauma reference group. These contrasting developments illustrate the divergent trends between time-based and non-time-based monitored time-sensitive diseases. We attribute these differences to the potential influence of the policy spotlight effect. The two diseases, AIS and STEMI, which are under the spotlight effect of policy, have shown an increase in the overall medical orders, diagnostic fees and medical expenses compared to the reference group, subsequently increasing the treatment indicators. As hypothesized, the septic shock group did not demonstrate significant changes in comparison to the major trauma reference group, underscoring the influence of time-based monitoring on these particular outcomes.
Discrepancy and unintended consequences
While both AIS and STEMI appeared to be influenced by the policy spotlight effect, they showed entirely opposite results in diagnostic indicators and transfer rates. This could potentially be attributed to the inherent differences between AIS and STEMI. For instance, only about 1%~2% of all AIS patients underwent primary treatment with intravenous thrombolytic agents, which is far lower than STEMI major treatment rate. Unexpectedly, in less policy spotlight affected groups such as septic shock and major trauma, despite significant decreases in primary diagnostic indicators, total medical orders, and diagnostic fees, there were unexpected reductions in 30-day and one-year mortality rates and overall medical expenses per event. These findings may align with the American 'less is more' initiative, suggesting that minimizing excessive diagnoses and unnecessary procedures could potentially enhance patient outcomes and yield cost reductions.
Policy implications
Currently, many emergency care policies implement time-based criteria 31–33, such as the UK’s 4-hour standard 33 and the four-hour rule 31. Australia's experience showed that an emergency care policy using time-based criteria can improve emergency congestion without increasing the rate of ED re-visits. However, in New Zealand, a policy in effect during 2006–2012 dictated that emergent patients must be hospitalized, transferred, or discharged within six hours of visiting the ED. After emergency care policy intervention, the length of ED stay decreased while the treatment outcomes of acute myocardial infarction, severe septic shock, and acute appendicitis did not improve significantly 32. Similarly, after the Canadian Emergency Observation Reduction Program implementation, the length of ED stay decreased while the treatment quality indicators for acute myocardial infarction, asthma, and upper limb fractures can only be treated in time for the above-mentioned time-sensitive diseases during the non-congested emergency period 34.
Emergency care quality is closely related to the practices of medical care providers 17. Policymakers and medical care providers must reconsider the conventional emphasis on time-based process indicators. This approach may have unintended consequences for diseases that are outside the "spotlight" of rigid time-sensitive evaluation. Instead, a broader and more nuanced evaluation of emergency care quality is needed to incorporate the complexity and ambiguity of various time-sensitive diseases that emergency care providers often manage 35. Therefore, we propose replacing time-based indicators with performance-based indicators, such as those exemplified in the NHS's Best Practice Tariff policy 36. This shift towards 'best practice' can create a more flexible approach that goes beyond merely relying on time-based or diagnosis-based practices 37. Such a transition is crucial, as an excessive reliance on diagnostic tests may lead to ED crowding, a decline in emergency care quality, and increased safety issues 38. The complexities of emergency care, including diverse patient origins and multiple routes to specialized units like stroke units, demand more sophisticated quality measures. These may encompass factors such as the presence of a consultant and the need for critical care admissions. In the case of specific procedures like diagnostic angiography and PCI (where indicated) within 72 hours of admission with NSTEMI, rigid time-based quality indicators may be especially inadequate. Performance-based measures, such as those used in the "Best Practice Tariff," reward best practices in care delivery and foster a higher standard of healthcare.
Strengths
To the best of our knowledge, this was the first nationwide retrospective cohort study using data on different CTSDs the effects of a policy exposed to an entire population. In the scenario that often presents challenges due to the lack of a control group, we identified differential impacts on various diseases through detailed analysis, establishing affected and counterfactual groups. This process allowed us to explore pre- and post-intervention effects, comprehending the policy's overall impact. This structure provides essential insights into the impact of hospital categorization on the processes and outcomes of emergency care. Further, it offers crucial empirical evidence that could be instrumental in refining policies and optimizing emergency care systems. This study explored emergency care providers' behaviors under time constraints and how they interacted with strict time-based quality surveillance indicators and “get with the guidelines” adherence.
Limitations
Because the study data were retrieved from a secondary dataset of insurance claims not a randomized controlled trial, this study has the following limitations: (1) our analysis lacked detailed information on time-related quality indicators, such as door-to-evaluation and door-to-treatment times measurement; (2) Our study's sample definitions were based on emergency primary diagnosis, making the emergency ICD code diagnosis potentially imprecise highlighting the need for a more comprehensive approach 39 (3) The effectiveness of emergency medical care often hinges on the cooperation between emergency physicians and the consulting physicians. This study is unable to dissect this relationship explicitly. Evaluating how emergency physicians and consulting physicians collaborate and coordinate is a key focus for future research.; (4) we need more qualitative research to elucidate the psychological mechanisms through which the policy spotlight effect influences emergency care providers' behaviour.