Title: Identification and Prioritization of Critical Factors Affecting the Performance of Hospitals, Using the Best-Worst Method: A prospective approach in Iran

Background: due to the numerous and ever-changing internal and external factors affecting health care performance, it seems essential to identify and prioritize them for managers and policy makers to make decisions easier, faster and more accurate. In this regard, the Best-Worst Method (BWM) is considered as a novel and innovative way to meet such an objective. Methods : At first, due to the large number of factors and indicators, the content validity ratio (CVR) was used to screen them. After that a Pairwise comparisons were conducted between the best and the worst criteria and the other one. Then Modeling the research problem through linear programming and finding the optimal weights. Results: the most effective internal factors from the experts’ viewpoint were the financial indicators such as “the ratio of personnel wage costs to the total revenues”, “the ratio of the costs of drugs and medical consumer goods to the total costs”. The external macro factors (driving forces) that were most influential were the “structure of the payment system and strategic service purchasing by insurance companies based on quality and price”, “the structure of tariffs” and the “fairness of the staff payment system”. Conclusion: The results showed that external factors had greater impacts on the performance of public hospitals than internal factors (efficiency and effectiveness). Therefore, when assessing the performance and improvement interventions of hospital, not only internal indicators should be measured, but also external factors, including political and economic policies and approaches governing the health system need to be taken into account.


Background
In a health system, performance refers to maintaining the patients' health while achieving business goals. Inefficiency, lack of meeting customer needs, and increased costs always threaten health systems (1). Health systems must improve their performance in order to appropriately respond to the challenges and changes of today's modern world (2). To this end, health managers and policy makers need a specific framework for evaluating the activities of the health system (1).
Performance evaluation is a constant challenge for managers and stakeholders, and is currently playing a crucial role in the success of organizations (3). Any organization needs a performance evaluation system to know about its status and quality of activities, especially in complex and dynamic environments. Hence, the lack of a comprehensive evaluation system is known as a symptom for a sick organization. The first attempt for evaluating hospital performance was made in 1859 by Florence Nightingale through considering mortality and infection rates (4,5). Since 2000, regular activities have also been started and developed to measure the performance of health organizations (6, 7).
Hospitals play a key role in health economy, accounting for 60% of total health expenditures (8). Therefore, measuring their performance seems essential. Performance indicators are also necessary for performance evaluation (9), because indicator-based evaluation helps to improve hospital performance (10). Meanwhile, identifying Key Performance Indicators (KPIs) for hospitals will help hospital managers to identify critical and recoverable points in the shortest possible time and at the lowest cost (11). As a performance-based decision-making tool, KPIs are vital for policy-makers and managers of organizations (12). Identifying and discovering performance indicators not only help to monitor, evaluate, and manage health systems to improve quality, efficiency, and effectiveness, but can also help to plan and develop organizational goals to deal with complex environmental changes (13,14).
Given the increasing complexity of hospital management, the need for scientific tools to evaluate their performance is felt (15). In this regard, selection of reasonable and appropriate indicators to reflect the objectives of evaluations should be done in such a way that it will adequately reflect the performance of the organizations (16). In addition, regular investigations and selection of performance indicators based on new evidence are necessary because of the continuity of and changes in hospital performance (17). Thus, due to the numerous and ever-changing indicators in the field of hospital performance evaluation, it seems essential to identify and prioritize them. Multiple criteria decision making (MCDM) plays an important role in solving multidimensional and complex problems (18). In this regard, the Best-Worst Method (BWM) is considered the most innovative method in multiple criteria decision making.
BWM is a novel approach to comparative multi-criteria decision-making that compares the best criterion with other criteria, and all the criteria with the worst one (19). The advantage of this method compared to many of the existing multi-criteria decision-making methods is that it requires fewer comparative data and leads to more stable results. The main reason for choosing the bestworst algorithm, compared with other multi-criteria algorithms, is the application of the least pairwise comparisons and thus, less likelihood of inconsistency (20).
Due to the novelty of this method, few studies have already used it in the field of health system. For example, Abu Hasham Abadi et al. used the BWM to develop a strategic planning framework for expanding the medical tourism industry in Yazd province (21). In her study, Mansoureh Roshan used this method to identify and prioritize the factors influencing outsourcing in Zeinabieh Hospital in Shiraz (22). In the present study, which is part of a project for a Ph.D. dissertation in Health Services Management, the BWM was used to identify and prioritize the key performance indicators for public educational hospitals in Iran.

Methods
In this research, a new technique called the best-worst method was used for solving multi-criteria decision-making problems (20). In multi-criteria decision-making problems, a number of options are evaluated and compared with a number of criteria in order to select the best option. In contrast, in the best-worst method, the best and worst criteria are firstly determined for decision-making, and in the next step, the best and worst criteria are compared with the other ones. Then, the problem is optimized and the weight of each criterion is determined. The weights of the options are obtained using various criteria. In the end, using the weights obtained, the options resulted from a set of criteria and options are calculated and the best one is selected. In this method, the inconsistency rate is also calculated. The main reason for preferring the best-worst algorithm to other multicriteria algorithms in the present study was the application of the least pairwise comparisons and consequently, the lower likelihood of inconsistency in comparison with other methods (20). In this study, the weights of the criteria were determined based on two features: probability and effect. The method consists of six steps: Step 1: Specifying a set of decision-making criteria. In this step, the set of indicators needed to make a decision is defined as follows: In this study, the factors affecting the performance of public hospitals were first extracted using a field review (12) and environmental scanning of the hospitals. Then, due to the large number of the factors and indicators affecting the performance of public hospitals, the content validity ratio (CVR) was used to screen the factors. In the CVR method, the content validity ratio was quantitatively calculated using the experts' comments. To determine the CVR, the experts were asked to examine each item based on a three-part spectrum comprising "effective", "effective but not necessary" and "not effective". The content validity ratio was calculated according to the following formula: In this equation, nE was the number of the experts who selected the "necessary" option, and N was the total number of the experts. As the panel had 32 members, the CVR higher than 0.29 was accepted (Appendix 1) (23).
Step 2: Determining the best (most important) and the worst (least important) criteria. At this stage, each expert selected the best and the worst criteria and indicators from his/her own point of view. No comparisons were made at this stage.
Step 3: Determining the superiority of the best criterion over the other criteria. The experts compared the criteria and indicators selected as the best in the second step with the other factors. This step and the fourth one dealt with the pairwise comparison approach by assigning a score of 1 to 9. The priority vector of the best indicator compared to the others was shown as .
Step 4: Determining the priority of the worst criterion among all the criteria by assigning a score of 1 to 9. The priority vector of other indicators compared to the worst one was shown as .
Step the maximum difference between the following equations needed to be minimized.
The result of minimizing the equations above was the formation of a function with restrictions as follows: To solve the preliminary problem, we first did the preparations for transforming it into linear programming. So the problem was formulated as follows: Step 6: Solving the linear programming problem and finding the optimal weights. At this stage, the indicators were evaluated and prioritized using the obtained weights. In this study, the LINGO software was used to analyze the data.

Findings:
Based on the experts' opinions, out of the 58 external driving factors and 33 internal ones affecting the performance of public hospitals, 18 and 11 factors were respectively selected using the CVR method ( Table 1).

Results of Screening with "Content Validity Ratio"
According to the results of the content validity ratio, the most effective internal factors from the viewpoint of the experts were the ratio of personnel wage costs to the total revenues, the ratio of the costs of drugs and medical consumer goods to the total costs, the ratio of total costs to available beds, the ratio of total costs to total income, the ratio of cost-effectiveness of any services, maintenance of human forces, the ratio of the doctors who used guidelines, and the compatibility of treatments to the patients' needs was the highest coefficient.
The driving forces and external macro factors that were most influential from the viewpoint of the experts included the structure of the payment system and strategic service purchasing by insurance companies based on quality and price, the structure of tariffs, increased costs of drugs and treatment, timely payment of hospital claims by insurance organizations, the health sector's share of the country's general budget, fairness of the staff payment system, budgeting system of hospitals, hospital requirements implement scientific and local guidelines agreed by the Ministry of Health and insurance organizations, and the progress of health information technology. (Table  1) General health insurance and a shift from employer-based to government-based insurance coverage (based on taxes) Payback paradigm shift from volume-based to value-based payment (quality, efficiency, safety, cost) in providing Value-Based Services Population aging and tendency toward the health system Prognostic medicine through studying human genomes

Health Megatrends
Of the four megatrends selected, General health insurance and a shift from employer-based to government-based insurance coverage (based on taxes) and the payback paradigm shift from volume-based to value-based payment (quality, efficiency, safety, cost) in the provision of health services had the highest coefficients.  As shown in Table 2, among the main dimensions, the economic and social ones obtained the highest and lowest ranks with the weights of 0.27 and 0.05, respectively. The internal dimensions (efficiency and effectiveness) were ranked third and fifth, respectively, in terms of their effects on the performance of public hospitals.
In the economic dimension, the most important indicators identified after ranking were the structure of the payment system, the inflation rate in the health sector, and the contribution of the out-of-pocket payments. In the legal dimension, the hospital budgeting system and the requirements for the implementation of scientific and local guidelines were identified as the main factors. In the social dimension, the changes in the disease burden pattern and the degree of equity in utilizing services were significant dimensions. It should be noted that the driving factors of increased risks and diseases caused by human environment pollution, downsizing the government according to various laws including Article 44 (development of privatization), and the advancement of health information technology (home care, telemedicine, distance education, and electronic health record) were introduced as the main drivers in the environmental, political and technological dimensions, respectively. In the area of the health system's megatrends, the valuebased payment paradigm and the public health insurance and government-based coverage were identified as the most important megatrends ( Table 2). Among the internal indicators, the waiting time in the emergency room (for triage or deciding on the patients' admission, discharge, or surgery), the ratio of the total costs to total revenues, the ratio of the doctors who used guidelines, and the ratio of cost-effectiveness of any services were the most important factors affecting the performance of public hospitals (Table 2).

Discussion
In order to evaluate and improve the performance of any organization, it is necessary to first identify the factors (internal and external) influencing its performance and determine the importance of each factor. Therefore, the present study dealt with identifying the key factors affecting the performance of public hospitals in Iran as well as the significance (weight) and rank of each. Ranking the indicators can help managers to only focus on key factors and indicators. Gupta and Barua argued that focusing on key factors would prevent waste of resources (24). Due to the space constraints, only the most important results (most effective dimensions and their important ranks and indicators) are discussed in the following.
According to the results of the BWM, the economic dimension with a weight of 0.27 was identified as the most effective on the performance of public hospitals in Iran. It was found out that in the economic dimension, the payment system structure had the greatest impact on the performance of the public hospitals. According to evidence, the payment system is one of the major issues in hospitals, because it is designed in such a way that it encourages service providers to provide a greater number of services (25). But in recent years, a shift from severity-based payments (volume, frequency, duration, and type) towards quality-based payments has begun, and this modification of the payment system is based on performance (P4P) (26,27), so that one of the megatrends in the Iranian health system is the paradigm shift from volume to value in the provision of health services (28). In the present study, "paradigm shift from volume to value" was selected as one of the sub-criteria affecting the performance of public hospitals, and obtained the fifth rank, indicating its great importance. This change could lead to the provision of quality services and reduction of costs (29). In this regard, one of the innovative methods in value-based systems is the strategic purchase of services or service packages (28,30,31). In Iran's health macro policies, issues such as modification of the quality-based payment system, increasing efficiency, creating fair income, and encouraging positive incentives for service providers have been mentioned (28). This suggests that if health managers and policy makers are seriously concerned about changing and reforming the payment system structure, hospital performance will be improved.
Another important criterion affecting the performance of public hospitals was the political dimension with a weight of 0.15. As the experts agreed, "downsizing the government according to different laws" was considered as the only key sub-criterion affecting hospital performance in this external dimension. Evidence suggests that government downsizing is one of the strategies considered by policymakers and managers, especially in the health sector, for organizational improvement and development (32). Downsizing is one of the most common interventions and an essential component of health system reform (33), because it can help improve the performance of public hospitals by reducing costs, reducing service delivery duration, increasing efficiency, enhancing skills, and focusing on main processes (34). One of the growing paths in this field is the collaboration of the public and private sectors in synchronizing the two sectors and increasing the value added of hospital services. This collaboration can greatly disentangle hospitals from upper organizations and strict administrative regulations, and create more flexibility in the structure, organization and performance of the management. This advantage, along with reduced financial dependence of hospital to the government budget, will give future hospitals more functional autonomy based on their goals (35)(36)(37). However, Kavosi et al. emphasized that deciding on the use of participatory models of the public-private sector was a complex and multi-criteria decision, all aspects of which needed to be addressed (38).
The main dimensions of efficiency and effectiveness, as the internal criteria affecting performance in this study, were ranked third and fifth, respectively. Among the sub-criteria of this dimension, "waiting time in emergency rooms" and "ratio of total costs to total revenues" were respectively ranked sixth and tenth, indicating the importance and great impact of these two indicators on the performance of public hospitals. As an important indicator of public hospitals, the "ratio of total costs to total revenues" indicated the importance of hospitals' financial performance affecting their efficiency. In this regard, Rahimi et al. selected it as one of the key indicators of the balanced scorecard financial indicator of Iranian hospitals. In their study, Rahimi et al. showed that the "ratio of total costs to total revenues "factors affected the other 16 indicators (73%), which is consistent with the results of the present study. Si et al. also described the role of financial indicators in improving hospital performance, which had complicated relationships with other indicators (39).
The results of this study also indicated that the "waiting time in emergency departments of public hospitals" had a significant effect on the effectiveness of hospital services. Given that evidence shows a long waiting time in Iranian hospitals, more attention should be paid to this issue. The results of a meta-analysis by Fazel Hashemi et al (2017) indicated that this indicator was higher in the emergency departments of Iranian hospitals than the national and international standards (40). Since a waiting list indicates the inefficiency of processes and influences the effectiveness and customer satisfaction (41), particular attention must be paid to the factors causing and decreasing it. The study by Rahimi et al. showed that the waiting list in the emergency rooms affected the 8 following indicators: the ratio of total income to total costs, average length of stay, bed occupancy rate, discharge with personal satisfaction, hospital infection rate, patient satisfaction, and number of complaints received from patients. However, the factors causing the formation of waiting lists in emergency rooms included: personnel absenteeism rate, staff turnover, length of patients' stay in the emergency room, and clinical errors (3). Therefore, monitoring and controlling the indicators and factors affecting the waiting time could reduce it and its destructive effects, which would ultimately lead to improved hospital performance. Another effective way to reduce waiting time in an emergency room is the use of a triage system. Choi et al. conducted a study to investigate the effect of "triage rapid initial assessment by doctor (TRIAD)" on the waiting time in the emergency department, and concluded that the mean waiting time and the process time decreased by 38% and 23% (42). Besides, using some methods like quality function deployment and lean production were effective in reducing the waiting time. Using the quality function deployment technique, Rahimi et al. (2007) presented a model for improving the quality of emergency services (43). Vashi et al. also used lean principles to reduce waiting times in the emergency room. The results of their study showed a decrease of 12.6 minutes at the "arrival to the physician" time (44).

Conclusion
This study has important implications for health system managers and public hospitals in Iran, because, contrary to previous studies that focused solely on the internal indicators of hospitals, the present research had a comprehensive approach to simultaneously identifying internal indicators and external factors affecting the performance of public hospitals in Iran. The results showed that external factors (economic and political ones) had greater impacts on the performance of public hospitals than internal factors (efficiency and effectiveness). Therefore, when assessing the performance and improvement interventions of hospital, not only internal indicators should be measured, but also external factors, including political and economic policies and approaches governing the health system need to be taken into account.

Declaration
This manuscript has not been published elsewhere and it has not been submitted simultaneously for publication elsewhere. *Ethics approval and consent to participate This study is approved by Shiraz University of Medical Sciences ethics committee with the ID number of IR.SUMS.REC.1396.S274. there was not any human involvement, And only database is used. *Consent for publication There was no difficulty in publishing the results. All the included databases and materials are available for public use.

*Availability of data and material
The data-sets analysed during the current study available from the corresponding author on reasonable request.

*Competing interests
Authors declare that there is no conflict of interest.

*Funding
There is no funding. *Authors' contributions PS designed the study, finalized the data synthesis and its overall methodology; KP and RH conducted the interviews, retrieved the data and prepared the initial draft of the article. PB contributed to data analysis and edited the article. NH improving the structure and conclusion of the paper. The study was supervised by PS. All authors read and approved the final manuscript. Acknowledgements