Measuring Efficiency of Public Hospitals in Iran: A Comparative Study Using Extended Data Envelopment Analysis, 2012-2016


 Background: Aiming to enhance quality of care and increase efficiency, public hospitals have undergone several reforms in the course of last two decades in Iran. This paper reports the result of a national research that aimed to measure the technical efficiency and productivity change of public hospitals during 2012-2016 in Iran. Methods: We used Extended Data Envelopment Analysis (Extended-DEA) (an innovative modification to conventional DEA) to measure technical efficiency and productivity of 568 public hospitals. Nationally representative data were extracted from the official annual health reports. Data were analysed using GAMS software 24.3. Results: The average efficiency score of all hospitals was 0.733. 10.1% of all hospitals were efficient while 2.68% of them were under 0.2. The Malmquist Productivity Index (MPI) progressed in 49.3% of hospitals, remained constant in 2.3%, while 48.2% of hospitals regressed during 2015-2016. The average of MPI was 1.07 over the period of analysis. Conclusions: Extra efforts seem to be essential to enhance the efficient use of resources and develop appropriate policy solutions and tools. In particular, to increase the return to scale, we advocate the merger of small-size district hospitals towards establishing bigger efficient hospitals in various geographical regions across the country.


Background
Efficiency is one of the main goals of the health systems worldwide 1,2 . Hospital expenditures represent around 30-50% of the total health expenditures in low-middle income countries (LMICs) 3,4 , hence assessing the efficiency and productivity of hospitals is of triumph importance for any healthcare system. Further, hospital is a very complex social organization that plays a significant role in the maintenance and promotion of social health 5 . Pursuing their goals in providing healthcare services to citizens, education and research, the ultimate goal of a hospital is to meet the societal health needs effectively and efficiently 6 . Let alone, the inappropriate use of new diagnostic and treatment technologies, ageing, escalating burden of chronic conditions, ever-increasing demand for healthcare services, plus the consequences of health professionals' errors and their negative side effects, have all imposed heavy costs on the health systems 7 .
Productivity is the sum of the effectiveness (doing the right things) and the efficiency (doing the things right) of an organization. Effectiveness means achieving organizational goals, while efficiency relates to achieving the desired outputs at a lower cost 8 , which indicates the ratio of outputs to inputs. The goal of efficiency is to maximize benefits against the costs incurred or to minimize costs for a given benefit. Various models and methods are used to measure the performance of organizations. These include DEA, Stochastic Frontier Analysis (SFA), and efficiency indicators 9,10 , among others, all of which operate based on two criteria of input (minimizing the use of inputs) and output (maximizing the output with a fixed input) 11 .
Many studies have been conducted on the efficiency measurement using the DEA approach in makers. Studies were combined using qualitative narrative synthesis. This paper reports data for each efficiency analysis on the: 1) evaluation context; 2) model specifications; 3) application of methods to test the robustness of findings; 4) presentation of results 20 . A systematic review conducted with another approach to search articles applying combined DEA and SFA in order to facilitate a common comprehension about the adequacy of these methods, defining any differences in healthcare efficiency estimation and the reasons that are behind this 21 .
Several studies have used DEA to assess the efficiency of hospitals, including in Iran 22 .
These studies have alluded to, although partially, the inefficiency of public hospitals, between 0.584 and 0.998 in Iran 17 . For instance, a comparison between the average length of stay and average occupancy rate indices in hospitals in Iran with other LMICs reveals the inappropriate utilization of existing resources 23,24 . Nevertheless, all studies used a single method for calculating the efficiency, and as a result, failed to provide a comprehensive picture of the hospitals' efficiency nationwide. No previous research has determined the current status of efficiency within all public hospitals in Iran 25 .
In the course of last two decades, public hospitals in Iran have undergone several reforms to enhance quality of care 26 and increase efficiency, i.e. decentralization, accreditation, and since 2014, improving hospitals' productivity through health transformation plan (HTP) 27 . Our team conducted a national research to measure the efficiency of the entire health system in Iran. This paper reports the findings of a study that aimed to measure the efficiency of all public hospitals during 2012-2016 in Iran.

Setting
Both private and public sectors contribute to providing hospital care in Iran. All hospitals are regulated under the supervision of the ministry of health and medical education (MoHME). In 2016, there were 921 active hospitals in the country, 80% of them were governmental and 20% were nongovernmental hospitals (Appendix 1), scattered across 31 provinces in Iran. In this research, we included governmental public hospitals, affiliated to the MoHME, and divided them into two categories: general and specialized hospitals. General hospitals were divided into three sub-categories: medical-non-educational, medical-educational, and medical-educationalresearch centers. Specialized hospitals were divided into eight sub-categories: Orthopedic, Accidents and Burns, Pediatric, Ophthalmology, Psychiatry, Gynecology and Obstetrics, Cardiology, Cancer and Oncology (This classification has been made by the MoHME). The average bed occupancy in public hospitals was 73% in 2014 28 .

Study design
This was a quantitative and descriptive-analytical study. Our sample included all governmental public hospitals affiliated to the MoHME in Iran. We extracted data from secondary databases linked to the MoHME's health information system (HIS). We measured the efficiency score, the MPI and provided the benchmark for each of the indicators. First, we conducted a literature review and used the classic DEA method to measure efficiency.
However, the initial results did not make sense for the research team. This was because the units that used only minimal inputs were efficient, while the health output had not been adjusted for both quality and equity aspects simultaneously. In other words, the reality of resource distribution, their case mix, and other contextual factors that may affect hospitals efficiency, were not taken into consideration in DEA conventional method. To overcome this challenge, we, in collaboration with a scholarly team in applied mathematics began a modification process, so-called extended-DEA to balance and rationalize the results. We will explain the three consecutive steps below: Step 1: definition of input-output indicators We conducted a qualitative analysis, i.e. literature review and collecting experts' opinions, to identify the input and output indicators. First, a scoping review of related studies identified a list of related indicators to the objectives of our research 29 . Second, we examined the existence of data associated with each indicator and the reliability of the data source, according to which, many indicators were excluded.Finally, the included indicators were reviewed and approved by an expert panel, comprising of the research team plus selected key informants in the field of heath management, policy and economics (Appendix 2).
Step 2: data collection and cleaning We used a checklist for data collection that was designed based on the input and output variables and the years studied. An Excel sheet was used to enter the data, acquired from the hospitals and workforce information database of the MoHME, for all public hospitals, as "Decision Making Units (DMUs)".
We then cleaned up the data to ensure the existence and accuracy of all data for each indicator per each DMU throughout all years of the study period. Irregular data was compared with other sources to ensure data integrity. We include all Iranian public hospitals, but due to limited number of input and output indicators; we exclude the DMUs without data for one indicator in one particular year (or years). Data collection and cleaning lasted six months. Following the opinions of selected key informants, we classified hospitals based on their specialty, teaching and non-teaching, as well as their performance indicators. To compare heterogeneous hospitals included in our study, we used the "level of specialty" variable, which let us classify similar hospitals in certain designated groups in a meaningful manner and conduct a meaningful fair comparison 30 ( Appendix 2).
Step 3: data analysis and modelling Each indicator was weighed and given a value using the standards set of the MoHME as well as the enjoying the views of an external advisory board. The more important an indicator was classified, the more influence it had on the efficiency score (Appendix 2).
DEA is a mathematically-based technique to determine the relative efficiency of a congruent DMUs.
Initially, in the DMU community, a point is determined and fixed as a benchmark for the DMU under evaluation, on the basis of alleviating the policies defined by the management.
Subsequently, relative efficiency of the DMU under assessment is calculated on the basis of benchmarking, which ranges between 0 and 1. The efficiency of a DMU under evaluation is signified by equating to 1, whereas, if this value is less than 1, it denotes the inefficiency of the DMU under investigation. Therefore, higher efficiency could be a sign of the DMU's better performance. In this article, we considered each hospital as a DMU, while hospitals were categorized into various specialty groups and EDEA models were independently implemented to each categorization.
We spouse to each hospital uses 4 inputs to create 7 outputs. we use the following symbols to show the values of inputs and outputs of the hospital j (j = 1,…, n). ij x : Value of ith input of hospital j , i 1,.., 4, j 1,..., n.  rj y : Value of rth output of hospital j , r 1,..., 7, j 1,..., n.  As described above, we determined the inputs and outputs for each hospital for modelling as follows: Since the fifth output (Average length of stay) is an undesirable output. We make the following changes to make it a desirable output.
(1) 5j y new 5j 1/ y  previous As we mentioned in method, given the definitions of each the input and output, the following constraints are taken for them based on the opinions of experts.
(2) On the other hand, the sixth output is expressed as a "percentage", so its value must always be between [0,100] . Therefore, the following constraints are considered in modelling.
(3) 315 5 j 6 j j 6 j j 1 j 1 0 y 100 The number of bed days is also dependent on the number of bed, which is why the following model constraints are considered in modelling. (4) The optimal value of the objective function of the model (5) can be denoted as a relative efficiency of hospital p. It is obvious that, if the optimal value of the objective function of model (5) is equivalent to 1, then hospital p (efficient). Similarly, if the optimal value of the objective function of model (5) is less than 1, then the hospital p can be called as being inefficient, so its coordinates of benchmark will be as the following: In order to calculate the progressive and unprogressive aspects of each of the hospitals on the basis of efficiency or performance, the Malmquist Productivity Index (MPI) has been computed.
This index is derived from the comparison of efficiency changes to technological modifications 31 , according to which, we divided hospitals into three groups: -Hospitals showing progress during (if MPI>1); -Hospitals showing regression (if MPI<1); and Hospitals whose performance remained constant during their period of study (if MPI=1).
The notation respectively. Two factors are effective in measuring productivity: A degree that indicates the improvement or deterioration in efficiency and is calculated as follows:  is efficiency at time t + 1 to real time t. In other words, the Catch-up Effect is the efficiency in the second to first periods.
() ii Frontier-shift Effect5: Calculates the boundaries of performance between the two periods and calculates the following.
The Malmquist Index is the ratio of efficiency changes and efficiency boundary changes that calculated as follows. (6)

MI 
For example, we solve the following model for

Results
We analyzed 568 hospitals within 2 categories and 11 subcategories. Initially, we began our study based on conventional DEA to measure the efficiency of hospitals. The primary results were difficult to interpret in the context of the Iranian healthcare system. Therefore, we started to categorized hospitals based on their specialty, the mix case, whether they are research oriented and/or train residents and fellows. To determine this criteria, we convened an expert meeting of some pioneers, including chancellors of medical universities, officials of the MOHME, some hospital managers and academics. Appendix 3 is a summary of categorized hospitals.
The descriptive statistics of inputs, outputs and explanatory variables are shown in Tables 1 and 2. We summarized the efficiency score and MPI of hospitals in Tables 3 and 4. Tables 5 and   6 show the efficiency score and MPI of the general and specialized hospitals during 2012-2016 in Iran. Finally, Table 7 presents the total inputs that need to be reduced and the outputs that need to be promoted for 2015.
Psychiatry Hospitals are kind of specialized hospitals, but we report all of the data and results of this group of hospitals separately to prevent the impact of their indicators on results. The standards and constrains of performance indicators in these hospitals are different from other specialized hospitals; for example, we set "X<3.5" for "Average length of stay" indicator, while the Average length of stay in Psychiatry Hospitals is more than Twenty days.   3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  SD 0.6 0.6 0.7 0.7 0.6 0.5 0.5 0.6 0.5 0.4 0.6 0.6 0.7 0.7 0. 5  Median  3  3  3  3  3  3  3  3  3  3  3  3  3  3    The average efficiency in specialized hospitals i.e. Cancer and Oncology, Orthopedic 20 Table 5 shows that, on average, 7 out of 315 general hospitals had the efficiency 21 scores of above 0.8, with the mean efficiency score of 0.357 over the analysis years.

22
The

38
On average, the lowest improvement was 0.768 in this index. In addition, 16 out of 22 39 hospitals had efficiency improvement over the analysis years.

41
The average efficiency score of orthopedic hospitals was 0.899 over the analysis

49
Among nine accidents & burn specialist hospitals, three hospitals had a score of

72
The average efficiency score in gynecology and obstetrics specialized hospitals was

78
The average efficiency score in nine specialized Cardiology hospitals was 0.769 79 (SD=0.16). On average, three hospitals were efficient (E> 0.8) and the range of 88 89  3  3  3  3  3  3  9  4  5  3 Cancer and Oncology There are two general ways to improve hospitals' productivity: reducing input 98 and/or increasing output. Table 7 presents the total inputs that need to be reduced 99 and the outputs that need to be increased to improve hospitals' efficiency in 2015.

124
It should be noted that a number of hospitals in this study were in the early years of 125 their establishment. Newly established hospitals operate inefficiently in the early 126 years due to shortcomings that they may experience at the outset of their activities.

127
The MPI indicates progress in the average efficiency score of these hospitals over the 128 period of analysis.

130
The average efficiency score of teaching hospitals varied between 0.354 and 1. On

142
The efficiency score of accident and burn hospitals in provincial centers and other

154
The variations' range (R) in specialized gynecology and obstetrics hospitals was 155 higher than other specialized hospitals (R=0.420), their efficiency score was low. This 156 could be due to the low bed occupancy rate of these hospitals (Mean <75%), which 157 might in turn be the result of the presence of similar wards in many public hospitals.

158
Logical reduction of these wards in general hospitals may enhance the efficiency of 159 these specialized hospitals.

161
Similar to other specialized hospitals, Cardiology specialized hospitals in the capital

167
While the average efficiency score in cancer and oncology specialized hospitals was

213
In the end, it should be noted that the efficiency score over the years under review 214 was not significantly altered, and most of the changes were related to Orthopaedics 215 and Accidents and Burns hospitals.

Rigor of study 218 219
This study began to use the Extended-DEA method as the most utilized technique to 220 measure hospitals' efficiency worldwide. Our findings were hard to interpret, so we 221 engaged with a scholarly team in applied mathematics to revise the method and measure efficiency 40 , so we classified hospitals in similar groups. We also brought 226 some qualitative variable as output indicators (Accreditation Degree of hospitals) in 227 our calculation, which is usually neglected in efficiency measurement studies. Last

231
Limitations 232 Despite advantages, our study had some limitations. Due to shortages in an 233 established monitoring system to collect the related data on hospital efficiency, 234 reliable and valid hospital data with enough input and output variables is not 235 available in Iran. Further, despite our efforts to obtain data form the MoHME, which 236 is the most reliable and available sources in Iran, there still remains some limits in 237 data credibility that might affect the reliability of our data source. Nevertheless, there 238 is no other source of data available to conduct analysis as in this study.

239
Our study could have benefited from some technical considerations to enhance