A Systematic Review of Hospital Efficiency and Productivity Studies: Lessons from Australia, UK and Canada.

doi:10.21203/rs.3.rs-123445/v1

Download PDF

Research Article

A Systematic Review of Hospital Efficiency and Productivity Studies: Lessons from Australia, UK and Canada.

https://doi.org/10.21203/rs.3.rs-123445/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Ageing populations, more expensive technology, growing rates of chronic disease and increasing consumer expectations are expected to lead to increased demand for health services and a rise in health expenditure within Australia. Productivity and efficiency analysis of Australia’s health system could provide valuable insight into the performance of the health system and assist stakeholders to reduce unnecessary growth in public hospital expenditure. This review describes efficiency and productivity analyses of hospitals in Australia, Canada and the United Kingdom.

Methods

We conducted a systematic literature review of efficiency and productivity analyses of hospitals in Australia, Canada and the United Kingdom. The search was conducted in two stages; (1) a search of the grey literature using a Google search engine; and (2) a traditional systematic review method search of academic databases. It is uncommon for grey literature to have abstracts, therefore, executive summaries, table of contents or subheadings were screened. Titles and Abstracts of journal articles were screened.

Discussion

Within Australia and key comparator nations, the number of efficiency and analysis studies is small. There is no clear consensus on the most suitable analysis technique to measure efficiency and productivity of hospitals. However, selection of inputs is similar across all studies identified in this review, consisting of measures of labour (most commonly relating to full time equivalent employees), goods and services (e.g. purchased consumables, such as drugs), and capital. Similarly, the majority of studies struggled to identify output measures that could capture improvements in patient outcomes, a key performance measure for any hospital. Instead, most studies utilised proxy measures relating to hospital throughputs (number of separations) or population health measurements. Of note, only one study demonstrated active engagement with the health sector in study development.

Conclusion

There is considerable scope for the further development of efficiency and productivity analysis techniques that can adequately capture relevant production factors, allow for robust comparisons across hospitals and time periods and which meaningfully engage with the health sector to inform improvements in efficiency and productivity.

Health Economics & Outcomes Research

Health Policy

Efficiency

productivity

health systems

Australia

Australia has a universal health care system that provides high quality care and is often ranked among the best in the world across a number of indicators (1, 2). In the coming years, an ageing population, more expensive technology and growing rates of chronic disease are expected to lead to increased demand for health services and a consequent rise in expenditure (3). This expected increase in health expenditure is likely to be particularly evident in the public hospital sector, with public hospitals the largest area of recurrent health expenditure in Australia. In 2015–2016 State, Territory and Commonwealth expenditure on public hospital services was $51.1 billion, a real growth of 4.4% from 2014–2015 and consistent with the average annual real growth over the decade. There are growing concerns that this rise in public hospital expenditure is unsustainable (4). Productivity and efficiency analysis of Australia’s health system could provide valuable insight into the performance of the health system and assist relevant stakeholders to arrest further unnecessary growth in public hospital expenditure.

Productivity and efficiency analyses measure the performance of ‘firms’ (whether these be factories or hospitals). Although often used interchangeably, productivity and efficiency are distinct (but related) terms. Productivity is defined as the ratio of outputs to inputs for a given firm and can be represented by a production frontier, with input on the (x) axis and output on the (y) axis. Larger values of the ratio of outputs to inputs are associated with better performance or productivity (5).

In his 1957 paper, Farrell provided seminal definitions of efficiency. Technical efficiency is defined as the production of the maximum amount of output from a given amount of input (or alternatively the production of a given output with minimum input quantities) given current technology. When a firm is ‘technically efficient’, it operates on the production possibilities frontier (PPF), (6) which is identified from the best performing firms. As such, efficiency has a comparative element, whereby firms are compared with an optimal production situation (the PPF). Allocative efficiency is similar to technical efficiency but places a cost on inputs or outputs. Allocative efficiency is defined as the input mix that minimizes cost, given input prices or when the output mix maximizes revenue, given output prices. Together, technical and allocative efficiency comprise overall economic efficiency.

As the number of inputs and outputs of a firm increases, measurement of its efficiency and productivity becomes increasingly complex. There are a number of approaches to measuring efficiency in hospitals, which have been utilized in Australia and internationally. These include least square econometric production models, total factor productivity (TFP) indices, data envelopment analysis (DEA), ordinary least squares (OLS) and stochastic frontier analysis (SFA) (5).

This review sought to understand how efficiency and productivity analysis could be applied to Australian hospitals. To further this understanding, the review also investigated efficiency and productivity analysis of hospitals in Canada and the United Kingdom, on the basis of their similarities to Australia (each are developed economies, provide Universal Health Coverage and are roughly of similar population size).

This reviewed specifically addressed the following question:

What methodologies have been employed to measure the efficiency and productivity of hospitals in Australia, Canada and the United Kingdom?

Search strategy and selection criteria

The literature search incorporated two different search strategies: (1) a search of grey literature sources using a Google search engine (Chrome) to access these sources; and (2) a traditional systematic review method search of academic databases.

As the hospital sector in Australia, Canada and the United Kingdom is largely Government funded, many studies of efficiency and productivity may fall into the category of ‘grey literature.’ Grey literature has been defined as ‘that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers’ (7). These documents are typically published via web-based resources and coverage in academic journals is limited (8). To date, no ‘gold standard’ method for grey literature searches has been developed (8). The authors identified one methodological study(8) that provided comprehensive details for applying systematic review search methods to grey literature and that adheres to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (9) and the methodology in this study was employed for this review.

Selection criteria

Selection criteria was defined by the researchers and are presented in Table 1.

Table 1

Selection criteria
Inclusion criteria	Exclusion criteria
Published in English	Not available in English
Current version of the document	The document was a draft version that has been replaced with a more recent document
Included information on an efficiency and/or productivity study of a public or private hospital or a health service or hospital district or similar (i.e. a network of organizations that provides or makes arrangements to provide hospital care to a defined population)	Did not include information on an efficiency and/or productivity study of a public or private hospital or a health service or hospital district or similar (i.e. a network of organizations that provides or makes arrangements to provide hospital care to a defined population)
	Efficiency or productivity studies of sub-hospital organizational units, e.g. surgical wards
All years of publication were included

(1) Grey literature

The authors used Google Chrome to search and access relevant grey literature sources, including the websites of Australian, Canadian and United Kingdom Government Health Departments, Statistics, Accounting, Taxation and Productivity Offices and the websites of health economics research centres. The grey literature search was conducted between May 15th and July 20th 2018.

Due to the mechanism by which Google searches and returns results, it is impractical and unnecessary to screen all returned results. Google uses algorithms to rank the importance of website pages relevant to search terms, with relevancy of results decreasing with each page return.(10) The researchers therefore screened only the first 10 pages that were returned for each search.

The following keywords and phrases were included in the search: Efficiency, Productivity, Hospital, Health, Australia, Canada, United Kingdom, England, Northern Ireland, Wales

The keywords were combined in different formats using OR and AND. An example search strategy used was as follows:

(efficiency or productivity) AND Australia AND Hospital

(2) Academic databases search

The second search strategy of academic databases (Scopus, Web of Science, MEDLINE) was conducted during the month of August 2018. The lead author searched titles, abstracts and keywords in databases to obtain peer review journals that met the inclusion criteria. The same keywords used in the first search strategy were used in the database search by combining different words using “OR”, “AND” and Truncation (*). A search strategy used in Scopus is presented in Table 2.

Table 2

Search strategy, Scopus
	Search Strategy	Results
1.	TITLE-ABS-KEY ( "efficiency" )	2,157,342
2.	TITLE-ABS-KEY ( "productivity" )	344,898
3.	TITLE-ABS-KEY ( "hospital" )	1,807,808
4.	TITLE-ABS-KEY ( austral* )	569,878
5.	( TITLE-ABS-KEY ( "efficiency" ) ) OR ( TITLE-ABS-KEY ( "productivity" ) ) AND ( TITLE-ABS-KEY ( “hospital” ) ) AND ( TITLE-ABS-KEY ( austral ) )	871

Data extraction

(1) Academic databases

The Title and Abstract of manuscripts were screened against the eligibility criteria. Manuscripts that did not address (either explicitly or implicitly) an analysis of the efficiency and productivity of hospitals or health service districts or similar in Australia, Canada or the United Kingdom were excluded. After title and abstract screening, the full texts were imported into Endnote and duplicates were removed.

(2) Grey literature

Many grey literature documents do not have abstracts, therefore, executive summaries, table of contents or subheadings were screened. Document details, including the source organization, title, date published and URL, were recorded in a Microsoft Excel spreadsheet. The final documents were downloaded in full to ensure they addressed the research questions.

The combination of the three search strategies resulted in a total of 29 documents.

Quality assessment

This study does not seek to determine a quantitative outcome, but rather explore the methodologies that have been employed to assess the efficiency and productivity of hospitals in Australia, Canada and the UK. As such, no risk of bias assessment was made and a quality consideration of the documents included would not have had an impact on the thematic findings.

Data analysis

Studies were assessed to determine the type of efficiency and productivity analysis employed and the inputs and outputs selected for inclusion in the analysis.

The PRISMA Flow Diagram shown in Fig. 1 summarizes the search process used in this review.

The review identified 31 documents that assessed the efficiency and productivity of hospitals in Australia, Canada and the United Kingdom. The below analyzes and compares these studies based on their country of origin, the type of efficiency and productivity analysis employed and the input, output and any other variables selected for inclusion in the analysis.

National measures of hospital performance

Within Australia, little work has been done to assess the efficiency and productivity of Australian hospitals. The Performance and Accountability Framework (the ‘Framework’) provides a national structure for reporting on public and private hospitals, and primary health organization performance (11). The Framework contains 48 indicators of health system performance, of which only four relate to efficiency. Data collection and reporting occurs for only two of these indicators and neither are a true measure of efficiency or productivity (12). Aside from the Framework, there are only a small number of studies (7) relating to the measurement of efficiency and productivity of Australian hospitals.

Similarly, within Canada, due to difficulties in data collection and recording, there is no national data available on the efficiency or productivity of Canadian hospitals. As noted by Ariste and Yu, the Canadian national accounting approach measures the total cost of inputs as total expenditures in publicly provided services, such as hospital care. This effectively means that total input costs are taken as total output, and it is therefore not possible to calculate the total factor productivity of the health sector (13, 14). Ariste and Yu therefore employ an episode–based direction measurement method that closely resembles the concept of output in the system of national accounts. This methods attempts to decompose expenditure changes across a given time period into a price change and a quantity change. Quantities measured are hospital inpatient and day surgery services with cost shares derived by using either the number of episodes for each major clinical category (MCC) or case mix group (CMG), or the number of episodes for each International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Canada clinical Chap. (13).

Despite the difficulties noted by Ariste and Yu, 12 studies have analysed the efficiency and productivity of Canadian hospitals. In addition, a number of studies have explored the measurement of health care sector output in Canada, including quantifying the cost function of various providers (14–17). Although these studies do not provide technical measures of efficiency and productivity and are therefore not further discussed in this review, they provide a useful exploration of the drivers of productivity in the health care sector and the difficulties in measuring the ‘true’ output of the health care sector, the improvement in length or quality of life in the population. A number of other studies also provide general discussion and overview of some of the challenges and key considerations in the measurement of productivity change in the Canadian health sector (18–20).

Unlike Australia and Canada, the United Kingdom has a well-developed national measure for assessing efficiency and productivity within the health system. The most significant body of work arises from the United Kingdom Office of National Statistics (ONS) and the Centre for Health Economics at University of York (21). The ONS provides measures of productivity for public service healthcare in the United Kingdom (and also England as a separate entity), (22) whereas the Centre for Health Economics considers only public service healthcare in the English National Health Service (NHS) (23). As both the ONS and the Centre for Health Economics investigate the productivity of public service healthcare, measures of input and output include primary care services (e.g. GPs) in addition to hospital care (22, 23). Each organisation provides a measure of productivity for the NHS as a whole by calculating the ratio of total quantity of healthcare output provided (adjusted for quality) with growth in the total quantity of inputs used (22, 23). However, as these productivity analyses relate to the whole NHS, as opposed to individual hospitals or Trust, they are considered outside the parameters of this review and are not further considered .

Analysis Technique

SFA is the analysis technique that has been used in the majority of efficiency and productivity studies of Australian hospitals (24–28). It is chosen for a number of reasons: it allows for measurement error, as opposed to attributing all the observed differences between hospitals to differences in technical efficiency; it can control for the influence of other unspecified environmental factors that may influence the relationship between outputs and inputs (24, 25, 27, 28); it provides a more conservative estimate of the scope to increase output (28); and it can identify the ‘true’ structure of the production function (24, 26). Whilst one study notes that the use of TFP indexing to measure the performance of government trading enterprises (which combines all outputs and inputs into a comprehensive measure of overall productivity) is increasing, TFP is not considered generally applicable to service provision, as it requires a price for each output and input and for many government enterprises it is not possible to identify output prices (29).

Studies in Australia that have adopted a DEA approach have done so as it provides a means to identify possible ‘peers’ or ‘role models’ by which benchmarks for performance can be established (29). This is particularly important for government agencies looking to control and standardise public expenditure on health services; it is interesting to note that of the two studies employing DEA in Australia, both were conducted by government agencies. Correspondingly, SFA is advantageous when it is the performance of the aggregate data set which is of interest, rather than any individual unit (24).

DEA, when used in Australia, was also selected for its inherent flexibility and its ability to handle multiple outputs where realistic price data for inputs and outputs is unavailable (24). However, as noted by Webster et al., a key drawback to the adoption of DEA is its assumption that there is no error in the data being studied (meaning that any identified inefficiencies are attributed to inefficiencies in the units under study, as opposed to random error) and the lack of a definite functional form encapsulating the production technology (24). This may be advantageous as there are fewer restrictions imposed on the model and less risks associated with mis-specified functional forms. But, input and output variables included in the model cannot be tested for significance, meaning that explanatory variables of little significance may be included in the model (24, 28).

In contrast to Australia, the majority of the Canadian studies identified utilise DEA, for a number of reasons: it accommodates multiple heterogeneous inputs and outputs to capture the complex nature of health service delivery (30–32), it does not rely on a priori assumptions in the specification of production frontiers (32) and the random error distribution (31); and it is less sensitive than SFA to underperforming outliers (31). No studies utilise SFA but three studies do utilise a Malmquist Productivity Index (MPI) in addition to DEA analysis (33–35). Like DEA, the MPI allows estimation of changes in overall productivity, but also allows decomposition into the efficiency change and technology change for each decision making unit over time (34). Asmild et al employ a slight variation of this approach, using the Malmquist Index of Asmild and Tam (36) that provides an overall measure of the movement of the frontier, as opposed to the standard approach that estimates the frontier shift specifically to each observation. This is advantageous as it can be applied to assess province specific frontiers for Ontario and New Brunswick, the provinces from which the sample of 114 public hospitals was drawn. These provinces differ substantially in their geography, Ontario being predominantly urban and New Brusnwick predominantly rural (33). This approach might therefore be also usefully applied within Australia, which has a similar geographic spread of hospitals.

Unlike Canada or Australia, DEA and SFA has had very limited use in assessing health system productivity and efficiency in England (37). This may in part be due to the number of governance models that have been place since the establishment of the NHS (38) that have shaped how efficiency and productivity is viewed. Following the introduction of an ‘internal market’ in 1991 in England, the official method adopted by the English Department of Health to promote efficiency gains was to undertake relative performance evaluation, whereby each the unit cost of were compared to those observed in other hospitals, with exogenous influences corrected for by using OLS methods. After correcting for these exogenous influences, residual unexplained cost differences were interpreted to represent inefficiency (39, 40). Reflecting this approach, a number of studies around this time utilise OLS (41). However, the use of this approach to make comparisons across organisations was later called into question, with some studies using SFA to raising concerns that residual unexplained costs differences should not be automatically attributed to inefficiency (39, 42). These are some of the few examples of SFA being used to assess productivity and efficiency in the United Kingdom.

As described above, the United Kingdom ONS provides measures of productivity for public service healthcare in the United Kingdom (and also England as a separate entity), (22) and the Centre for Health Economics at University of York have applied similar methods to individual hospitals and NHS Trusts (23). The Centre for Health Economics calculates the ratio of total quantity of healthcare output provided (adjusted for quality) with growth in the total quantity of inputs used and combines this with ordinary least squares regression to provide a measure of efficiency and productivity. This approach is touted as a mechanism to avoid some of the limitations of DEA whilst providing a means to investigate variations in hospital productivity (43). As outlined by Castelli et al the number of outputs produced by a hospital is usually far greater than the number of hospitals under consideration. DEA is only capable of dealing with multiple outputs up to a point determined by the number of organisations under consideration (43). To overcome this issue, Castelli et al draw on the prior work of Dawson (40) and Castelli et al (44) to propose an alternative approach to dealing with the problem of multiple outputs in DEA. This approach is an extension of the national accounting framework to measure changes in health productivity and involves the application of an explicit set of weights to combine diverse outputs into a single index, obviating the need for DEA (43).

Using this approach, Castelli et al construct productivity indexes for the financial years 2008/09 and 2009/10 for English NHS Hospital Trusts (43). The inputs and outputs used in the construction of these productivity indices is extensive and draws on a number of input and output factors to construct composite input and output indices. This approach has been highly influential in considering the productivity and technical efficiency of hospitals across the UK and has been adopted and modified by a number of subsequent studies. A similar approach is employed in a subsequent study by Bojke et al that measures NHS productivity growth as a set of paired year-on-year comparisons from 1998/1999–1999/2000 through to 2012/2013–2013/2014 (45). Aragon et al broaden the definition of outputs to more fully reflect the array of diverse activity provided by hospitals (46).

Outside of England, a small number of studies adopt DEA, often in combination with Malmquist Indices. Malmquist approaches are often employed in Scotland, as the Scottish NHS is more socially driven as compared to the current governance model of the English NHS. Because of this framework, typical economic specifications such as cost minimisation or profit maximization are not appropriate (47) (Scottish hospitals are funded via block contract). The Malmquist index, which does not require input or output prices, is therefore a common approach in this health sector.

Table 3 identifies the analysis technique by country for each study identified in this review.

Table 3

Analysis techniques utilised in efficiency and productivity studies of Australian, Canadian and United Kingdom hospitals
Analysis Technique	Country	Authors (year), publication
Average cost per case	Australia	Braithwaite et al (2006), Health Services Management Research
DEA	Australia	Steering Committee for the Review of Commonwealth/State Service Provision (1997), Australian Government
DEA	Canada	Chowdhury and Zelenyuk (2016), Omega
DEA	Canada	Allin et al (2016), Health, Economics Policy and Law
DEA	Canada	Allin et al (2015), Healthcare Policy
DEA	Canada	Abeney and Yu (2015), Lakehead University
DEA	Canada	Fixler et al (2014), Health Services Management Research
DEA	Canada	Canadian Institute for Health Information (2014), Canadian Government
DEA	Canada	Gruca and Nath (2001), Healthcare Management Science
DEA	Canada	Biodeau et al (2009), Applied Economics
DEA	Canada	Bilodeau et al (2004), Journal of Productivity Analysis
DEA	Scotland	Ferrari (2006), The Service Industries Journal
DEA	Northern Ireland	McCallion et al (2000), Applied Economics
DEA	Scotland	Hollingsworth and Parkin (1995), Mathematical Medical and Biology: A Journal of the IMA
DEA Malmquist Index of Asmild and Tam	Canada	Asmild et al (2013), journal of Productivity Analysis
DEA Malmquist Productivity Index	Canada	Chowdhury et al (2014), International Journal of Production Economics
DEA Malmquist Productivity Index	Ireland	Gannon (2008), Applied Economics Letters
DEA SFA OLS	Australia	Webster et al (1998), Australian Government
Malmquist Productivity Index	Canada	Chowdhury et al (2011), International Journal of Productivity and Performance
Malmquist Productivity Index	Scotland	Valdmanis et al (2017), Health Services and Outcomes Research Methodology
Malmquist Productivity Index	Scotland	Maniadakis et al (1999), Healthcare Management Science
OLS	Scotland	Scott and Parkin (1995), Health Economics
Productivity ratio	England	Aragon et al (2017), PLOS ONE
Productivity ratio	England	Castelli (2015), European Journal of Health Economics
SFA	Australia	Yong, Karen and Harris, Anthony (1999), Centre for Health Program Evaluation
SFA	Australia	Wang, Zhao and Mahmood (2006), IZA Discussion Paper Series
SFA	Australia	Gabbitas and Jeffs (2008), 30th Australian Health Economics Conference
SFA	Australia	Productivity Commission (2009), Australian Government
SFA	Scotland	Ferrari (2006), Applied Economics
SFA	England	Street and Jacobs (2002), Applied Economics
SFA	England	Street (2003), Health Economics

Outputs

A hospital’s performance should ideally be measured in terms of patient outcomes, the improvement in physical and emotional wellbeing relative to what would otherwise be the case if they had not sought treatment from a hospital (48). Within Australia, there is limited data available to directly measure the changes to patient health outcomes (28). As such, studies within Australia have largely employed proxy measures to indirectly account for changes in health outcomes. These proxy measures are usually measurable hospital outputs, such as number of separations,(25–27, 29, 49) number of occupied bed days (24) or occasions of service (24, 26). A number of Canadian studies also employ similar proxy outcomes, relating to discharges, inpatient days, outpatient and surgical visits (30, 32–35, 50–52).

Occasionally, ‘quality of care’ is also used as a proxy measure for health outcomes, with one study in Australia including risk-adjusted mortality ratios for each hospital in the analysis to provide some measure of hospital quality (28). Similarly, Abeney and Yu in their assessment of the efficiency of Canadian health provinces select the median wait time between various treatment phases and the median wait time for access to medical imaging resources as their output measure (53).

There are studies in Canada and the United Kingdom, however, which have developed methods that better approximate the changes to patient health outcomes. Three studies in Canada use provincial health regions as their decision making unit, rather than individual hospitals, and this has allowed these studies to incorporate a measure of patient outcomes into the efficiency and productivity analysis (31, 54, 55). For each study, the selection of outputs was determined in consultation with a range of stakeholders in these health regions, including senior health ministry officials. The consensus was that any measure of efficiency should be measured against the objective of providing access to effective care to those who are sick or otherwise in need of care. To measure this objective, each study selected an output variable of the reduction of Potential Years of Life Lost (PYLL) for causes of death that are considered amenable to health system intervention, such as sepsis, pneumonia, colorectal cancer, breast cancer in women, hypertensive diseases, asthma and most other respiratory diseases, renal failure, pregnancy and childbirth. In each case, rates of PYLL per 100,000 population for the latest available time period were age-standardised to account for the different age structures across regions. Estimates were based on Canada’s vital statistics database held at Statistics Canada (31, 54, 55).

Utilising the health region as the decision making unit in the analysis facilitated the use of these health outcome measures, as apart from a few exceptions, population health data was available at the regional level. Reduction in PYLL can still be considered a proxy for individual health outcomes, as it is effectively a measure of premature mortality and does not address quality of life; reduction in PYLL may also be influenced by factors outside the health system, including social determinants of health (e.g. socioeconomic status) and other factors, such as smoking rates. To address this, a number of these ‘environmental factors’ were included as inputs in the DEA analysis and in a subsequent regression analysis to identify the factors that help explain variation in inefficiency across regions. Factors not significantly associated with the outcome measure were included as factors only in the subsequent regression analysis.

Castelli et al include a similar measure for patient outcome in their construction of productivity indexes for English NHS Hospital Trusts (43). The output index includes a measure of quality, by constructing the equivalent of a Quality-Adjusted Life year (QALY) profile for patients allocated to each Healthcare Resource Group (HRG). Although this equivalent QALY profile is still somewhat a proxy measure of patient health outcomes, as it relies heavily on average measures and standardisation across age and gender, it is a significant step forward in accounting for health outcomes as a measure of hospital output and an approach which has not yet been adopted in Australia. Given that across Australia most states have adopted a similar organisational structure to United Kingdom and Canadian health regions, with individual hospitals and related health service providers in a given geographic region under the auspices of a Hospital and Health Service or Local Health Network, it may be worthwhile to consider similar analytic approaches in the Australian context.

Further confounding the selection of an appropriate output measure is the casemix – the mix of types of patients - treated by the hospital. Hospitals with a casemix of relatively ill or complicated patients (such as those with multiple morbidities) will likely require a higher level of input to produce the same number of outputs if outputs are measured in aggregated terms such as number of bed days or number of patients treated (i.e. such patients will have higher treatment costs). Within Australia, the introduction of casemix funding and diagnosis-related groups (DRGs) (a means of grouping patients based on the type of treatment received) (56) has provided a mechanism to account for variations in hospital casemix when conducting efficiency and productivity analyses.

Of the efficiency and productivity analyses of Australian hospitals only two eschew a casemix adjusted output. Gabbitas et al estimate a series of three input stochastic frontier models for public acute care hospitals in each State and Territory for the period 1996–1997 to 2005–2006 (27). They use a single output, the number of casemix adjusted separations from each State and Territory’s public acute care hospital system. While the separations are casemix adjusted at some point during data collection, the output used in the study is the total number of separations, with no attempt to weight this output variable with some measure of disease complexity, such as DRGs.

An Australian Bureau of Statistics study measuring the efficiency and productivity of Australian private hospitals also does not employ a casemix approach, but does make some attempt to account for the confounding nature of patient complexity (24). The study uses output measures based on the number of acute care and surgery inpatient occupied bed days, occasions of service and acute care separations. This approach was adopted after difficulties in applying disease-costing weights to separation data (the level of aggregation in grouping diseases was too high to effectively reweight the raw separations) and attempts to provide some adjustment for disease severity. The study goes on to acknowledge the potential of DRGs to effectively account for hospital casemix.

Issues with aggregation of disease groupings remains a challenge even when DRGs are utilised. A DEA study of the efficiency of 109 hospitals in Victoria using data for the period 1994-95 utilised two output measures: unplanned re-admissions rate (an imperfect proxy for the quality of care); and the number of patients treated by each hospital expressed in terms of weighted inlier equivalent separations (WIES), aggregated into three categories based on the degrees of complexity of each WIES. The WEIS are said to be aggregated into only three categories due to difficulties in incorporating the over 500 DRG variables into the DEA analysis (29).

Similarly, the Productivity Commission analysis of 508 public and private acute hospitals in Australia aggregated output variables into five major diagnostic categories (MDCs): acute separations; pregnancy and neonate separations; mental and alcohol separations; other seperations; and endocrine, nutritional and metabolic diseases and disorders.(48) Similar to the above study, this aggregation occurs due to concerns that using the full 23 MDCs would represent too many variables in the SFA analysis, particularly when more complex functional forms are employed (48).

Wang et al also acknowledges that as the number of diseases and conditions for which patients seek treatment is large (DRG categories), some degree of aggregation of hospital outputs is necessary to avoid running out of degrees of freedom in the efficiency analysis.(26) In their investigation of the efficiency of New South Wales hospitals using SFA, the authours use an ‘inpatient service index, incorporating the number separations and an index for the DRG diagnostic category to account for casemix variation (26). Yong et al employ a similar approach utilising WIES in their estimate of the efficiency of public hospital in Victoria by SFA (25).

Unlike Australia, Canadian hospitals are funded under a global budget model, rather than a casemix approach. Under global budgets, a fixed amount of funding, typically based on factors like historical funding amounts or population, is provided to a hospital or other health care provider and that provider then delivers services to patients for a fixed period of time (usually one year) (57).

Despite this, a number of methodologies have been developed to adjust patient flows for casemix, including Case Mix Groups (CMD) methodology (32, 34) or Resource Intensity Weights (RIW) (30, 33, 50) or Expected Length of Stay (ELOS) (30). The Case Mix Groups methodology identifies clusters of acute-care inpatients with similar clinical and resource-utilization characteristics and adjusts for various factors—such as patient age, sex and comorbidities (the number of conditions a patient has beyond the primary reason he or she was admitted into a hospital)—to account for how they may influence the costs of hospital stays (32). RIW is a relative value measuring the total patient resource use compared with average typical acute inpatients. ELOS is the average acute length of stay in hospital for patients within the same CMP, age category, comorbidity level and intervention factors (58).

Hospital payment arrangements differ significantly across the United Kingdom, with two distinct models of governance in place. In England, funding commissioners contract with providers to deliver care, with patients empowered to exercise choice between providers (38). Reimbursements for care provided is made at nationally determined DRG rates (known as Health Resource Groups (HRGS) in the UK) (59). Within England, HRGs are used to adjust for casemix in efficiency and productivity studies. Castelli et al in their construction of productivity indexes for English NHS Hospital Trust draw on a number of input and output factors to construct composite input and output indices, including an adjustment for casemix through the use of HRGs (43) and this approach has been used in a number of subsequent studies (45, 46).

Within Scotland and Wales regional health boards are responsible for providing primary and secondary care to their communities and are funded to deliver this care. This is also largely the case in Northern Ireland. As with Canada, DRGs are not utilised in these countries, with payments to hospitals and other health service providers in the form of block contracts. Given these differences in funding models, casemix adjustment approaches are rarely employed in studies outside England.

Inputs

Inputs in efficiency and productivity analysis are generally broken down into labour, materials and capital. The degree of further disaggregation across these categories is variable. Some studies distinguish between the various types of labour – nursing staff, allied health staff, administrative staff, etc. (24, 25, 28) - while others simply use an aggregate measure of total employment (27) or a crude split into medical and non-medical staff (26, 29). Of note, the few studies that assess the efficiency of Canadian provinces, rather than individual hospitals (31, 55, 60) also employ input variables relating to the type of services and the heath service providers in the region, such as Hospitals, Other Institutions and Public Health Services.

Within Australia, the treatment of capital can vary dependent on the funding arrangements of the hospitals under analysis. Yong et al consider that capital-related data such as depreciation and interest are poor estimates of the price of capital, as the Victorian hospitals analysed in their study do not own their buildings and assets and are not generally depreciated in public hospital accounting systems, nor do they pay interest (25). Gabbitas et al, however, include real capital services in their analysis of public acute care hospital in each Australian State and Territory (27).

The 1997 Steering Committee for the Review of Commonwealth/State Service Provision use an aggregate measure of non-labour costs in their efficiency analysis of Victorian public hospitals that excludes capital stock, due to limitations in data availability (29). Similarly, other studies report difficulties in adequately measuring capital (24) and instead use proxy measures, such as number of beds, where more extensive capital measures are unavailable (28).

The treatment of material costs within Australia also varies. Some studies exclude completely (25), assuming that the purchasing power of hospitals means that the price of materials is the same across hospitals and can thus be reasonably excluded from the analysis (26); others use a nominal or total expenditure on medical and drug supplies (27, 28). Thus, the availability of accurate data for material and capital assets is often a significant challenge when examining the efficiency and productivity of hospitals.

Table 4 identifies input and output variables used for each study, grouped by country.

Table 4

Inputs and output variables utilized in efficiency and productivity studies of Australian, Canadian and United Kingdom hospitals
Hospitals; number of observations; year	Inputs	Outputs	Authors (year), publication
*Australia*
Victorian public hospitals; 109; 1994-1995	FTE non-medical staff FTE medical staff Non-salary costs	WIES with intensity rate < 0.2 (Y1) WIES with intensity rate > 0.2 and , 0.4 (Y2) WIES with intensity rate > 0.4 (Y3) Inverse of the unplanned re-admission rate	Steering Committee for the Review of Commonwealth/State Service Provision (1997), Australian Government
Private hospitals; 301 (cross-section), 280 (panel); 1994-1995 (cross-section), 1991-992 – 1994-1995 (panel)	SMO (FTE) VMO ($ contract value) Nursing staff (FTE) Other staff (FTE) Beds Materials (non-labor costs) Admissions Total staff (FTE) Total staff (labor costs, $) Parametric (SFA and OLS): Beds Capital stock Materials (non-labor costs) Materials (including VMOs) Total staff (total FTE) Total staff (labor costs)	Acute care inpatient days (psychiatric care inpatient days, rehabilitation days, medical care days) Surgery inpatient days (advance surgery days, surgery days, minor surgery days, obstetrics days) Non-inpatient occasions of service Nursing home type inpatient days Surgical procedures Acute care inpatient separations Accident/emergency Composite output (of the above) Total inpatient revenue Parametric (SFA and OLS): Revenue Composite output (occupied bed days)	Webster et al (1998), Australian Government
Victorian public hospitals; 35; 1994-1995	Average medical, nursing, administration/clerical, hotel and allied, medical support and RMO salary per E.F.T	Weighted Inlier Equivalent Separations Total Operating Expenditure	Yong, Karen and Harris, Anthony (1999), Centre for Health Program Evaluation
New South Wales public hospitals; 114; 1997-1998	Average salary of medical officer and visiting medical officers Average salary of non-medical labor input	Inpatient Service Index (the proportion of a hospital’s separations times a weight corresponding to the average length of stay of separations with AN-DRG) OOS Average Available beds Same day separations % total separations Average length of stay of acute episodes Cost per Outpatient OOS Cost per emergency OOS Presentations to emergency department	Wang, Zhao and Mahmood (2006), IZA Discussion Paper Series
Teaching hospitals in New South Wales and Victoria; 20; 1991-1992 to 1996-1997 (NSW) and 1991-1992 – 19995-1996 (Vic)	-	Cost per DRG-weighted patient (NSW) WEIS (Victoria)	Braithwaite et al (2006), Health Services Management Research
State and Territory public acute hospitals; 1996-1997 to 2005-2006	FTE employment in public acute care hospitals Real capital services Real medical supplies	Number of casemix-adjusted separations	Gabbitas and Jeffs (2008), 30th Australian Health Economics Conference
State and Territory public and private acute hospitals; 508; 2006-2007	Number of FTE nursing staff Number of FTE pathology and radiology staff Number of FTE domestic, administration and other staff Expenditure on medical and surgical supplies Expenditure on pharmaceutical supplies Other input expenditure (e.g. clerical, housekeeping) Number of beds Patient comorbidity (Charlson score) Socioeconomic status of patient (SEIFA scores) Gender Indigenous status Age profile Remoteness of usual place of residence	Casemix-adjusted separations grouped into four categories based on the Australian system of major diagnostic categories: acute separations; pregnancy and neonate separations; mental and alcohol separations; other separations Risk-adjusted mortality ratios (the ratio of observed mortality rate divided by the predicted mortality rate)	Productivity Commission (2009), Australian Government
*Canada*
113 acute-care hospitals in Ontario; 2003 and 2006	Administrative staff hours Nursing hours Staffed beds Medical-surgical supplies costs Non-medical supplies costs Equipment expenses	Ambulatory visits	Chowdhury and Zelenyuk (2016), Omega
89 health regions; 2007-2009	Inputs – spending per capita ($) Hospitals Prescription drugs Physicians Residential care facilities Community nurses Inputs – environmental adjustors Education (per cent with high school or more) Recent immigrants (per cent) Non-Aboriginal (per cent)	Output – age standardised, per 100,000 population PYLL from treatable causes (before age 80) PYLL from treatable causes (before age 75) PYLL from treatable causes (before age 85) Mortality from treatable causes (before age 80) Survival rate from treatable causes (before age 80)	Allin et al (2016), Health, Economics Policy and Law
89 health regions	Spending per capita $ Hospitals Prescription drugs Physicians Residential care facilities Community nurses Inputs - environment Education (% with high school or more) Recent immigrants (%) Non-Aboriginal (%)	PYLL from treatable causes (before age 80), per 100,000 population, age standardized	Allin et al (2015), Healthcare Policy
Canadian provinces; 2001-2010	Capital: Residential care facility beds Labour: Pharmacists, Nurse practitioners, Specialists, Medial laboratory technologists General practitioners Machine:, CT scanners MRI For the expenditure input variables, the following proxies are used: Capital: Hospitals, Other institutions Other capitals Labour: Physicians, Other professionals, Administration Machine: Other health spending Services: Public health, Drugs	Median waiting time between General Practice visit, Specialist visit and Treatment Median Weeks Waiting Time for Computed tomography scanner, Magnetic Resonance Imaging and Ultrasound	Abeney and Yu (2015), Lakehead University
154 acute care hospitals; 1 April 2005 - 31 March 2006	Service Recipient Workload Units: e time spent by Unit Producing Personnel performing service recipient activities of the hospital. Bed Days staffed and in operation: (bed days) measures the calendar days that beds and cribs are available and staffed to provide services to inpatients and residents. It is a measure of the hospital’s capital input.	Inpatient/Resident Discharges and Deaths. Outpatient visits Inpatient surgical visits Outpatient surgical visits Resource Intensity Weight: resource allocation metric for estimating the costs of acute inpatient and day surgical cases, using clinically homogenous groups Expected Length of Stay: represents an estimation of the typical length of stay for acute inpatient cases.	Fixler et al (2014), Health Services Management Research
113 acute care hospitals in Ontario; 2002-2006	Human resources including nurses and administrative workers measured in FTE hours Purchased services and supplies including medical/surgical supplies and nonmedical/surgical supplies measured in dollars Number of staffed beds as a proxy for capital.	Model 1 used unadjusted inpatient days and outpatient visits, Model 2 used case-mix adjusted weighted inpatient days and outpatient visits as outputs Model 3 included unadjusted inpatient days, outpatient visits and the case-mix index as outputs.	Chowdhury et al (2014), International Journal of Production Economics
Canadian health regions	Spending per capita $ Hospitals Prescription Drugs Physicians Residential Care Facilities Community Nurses Inputs - Environmental Factors Education (High School or More) (%) Recent Immigrants (%) Non-Aboriginal Persons (%)	PYLL From Treatable Causes (Before Age 80), per 100,000 Population, Age-Standardized	Canadian Institute for Health Information (2014), Canadian Government
114 public hospitals in Ontario and New Brunswick; 1998-2004	FTE nursing hours by the five categories of nursing for each acute care hospital in each of the provinces	Resource Intensity Weight-adjusted discharges	Asmild et al (2013), journal of Productivity Analysis
113 acute care hospitals in Ontario; 2002/2003 to 2005/2006	Human resources including FTE hours for nurses and administrative workers Purchased services and supplies including medical/surgical and non medical/surgical supplies Number of staffed beds as a measure of capital Total equipment expense as a proxy of service-mix	Inpatient and outpatient volume	Chowdhury et al (2011), International Journal of Productivity and Performance
168 community general hospitals in Ontario; 1986	Nursing (FTE) Ancillary services (FTE) Administration (FTE) Services and supplies including drugs and medical-surgical supplies ($00000’s CN) Total beds	Inpatient cases weighted by Resource Intensity Weights Weighted outpatient visits Long-term days of care	Gruca and Nath (2001), Healthcare Management Science
121 short-term hospitals in Quebec; 1981/82 – 1992/93	Total number of hours and expenses on labor Supplies expenditure (medical, administrative, maintenance, security and capital maintenance) Food and meals prepared for inpatients Drugs Energy	Inpatient days Outpatient clinic visits Laboratory exams performed for pay (for non-hospitalized patients) Laundry and cafeteria services (for non-hospitalized patients) Teaching	Biodeau et al (2009), Applied Economics
121 short term hospitals in Quebec; 1981/82 – 1992/93	Total number of hours and expenses on labor Supplies expenditure (medical, administrative, maintenance, security and capital maintenance) Food and meals prepared for inpatients Drugs Energy	Average number of patient days Average number of clinic visits Average number of patient days Average number of clinic visits	Bilodeau et al (2004), Journal of Productivity Analysis
*United Kingdom*
43 general acute care Scottish hospitals; 2003-2007	Doctors (physicians and dentists) FTE Nurses (nurses in all categories including nurse trainees) FTE Other labor (all other labor inputs) FTE Staffed beds (a proxy for capital)	Inpatient cases (inpatient elective cases and inpatient emergency cases) adjusted for Healthcare Resource Group categories Outpatient and short stay patients (inpatient day cases, clinic attendances and emergency ward attendances)	Valdmanis et al (2017), Health Services and Outcomes Research Methodology
English NHS Trusts; 2010/11 – 2012/13	NHS Labour (Direct) Agency Labour Intermediate goods and services Capital	Elective and day cases Non-Electives A&E Chemo/Radiotherapy & High Cost Drugs Community Care Community Mental Health Diagnostic Tests Hospital/Patient Transport Scheme Other NHS Activity Outpatient Radiology Rehabilitation Renal Dialysis Specialist Services	Aragon et al (2017), PLOS ONE
English NHS Hospitals; 2008/09 – 2009/10	NHS labour Agency labour Intermediate goods and services Capital	Elective and day cases: Number of patients Mean 30-day post discharge survival rate Mean life expectancy in years 80th percentile waiting times Non-electives: Number of patients Mean 30-day post survival rate Mean life expectancy in years Outpatient volume of activity	Castelli (2015), European Journal of Health Economics
Acute hospitals in Ireland; 1995-1998	Average number of beds per year Number of people employed in each hospital (FTE)	Inpatients (total number of discharges and deaths) Outpatients (total yearly number of attendances) Both adjusted for casemix using DRGs	Gannon (2008), Applied Economics Letters
53 acute Scottish hospitals; 1991/92 - 1996/97	Total capital charges (£000) Medical staff FTE Nursing staff FTE Other staff FTE Total number of beds.	Inpatients ( surgery, medical other) Day cases and day patients.	Ferrari (2006), The Service Industries Journal
52 acute Scottish hospitals; 1991/92 to 1996/97	Total capital charges Medical staff FTE Nursing staff FTE Other staff FTE Total number of beds	Total number of cases weighted by casemix	Ferrari (2006), Applied Economics
217 English NHS hospitals; 1995-1996	Average Expected costs	Average Actual costs	Street and Jacobs (2002), Applied Economics
226 English NHS hospitals; 1995-1996	Average Expected costs	Average Actual costs	Street (2003), Health Economics
23 Northern Island Hospitals; 1986-1992	Nursing staff Administrative staff Ancillary staff (measured by full time equivalent staff members) Specialists Bed complement	Total number of inpatients and outpatients	McCallion et al (2000), Applied Economics
75 Scottish hospitals; 1991/92 – 1995/1996	Admissions for stroke Admission for fractured neck of femur Admission for myocardial infraction	Standardised survivals after admission for stroke Standardised survivals after admission for fractured neck of femur Standardised survivals after admission for myocardial infraction	Maniadakis et al (1999), Healthcare Management Science
75 acute Scottish hospitals; financial year 1992-93	Average number of staff beds Total number professional, technical, administrative and clerical staff Total of junior and senior non-nursing medical and dental staff (staff number measured as the average effective staff number for the year Cost of drug supply Capital charge	Acute in-patient days (medical) Acute in-patient days (surgical) Accident and emergency attendances Out-patient attendances Obstetrics and gynaecology in-patient days Other speciality in-patient days	Hollingsworth and Parkin (1995), Mathematical Medical and Biology: A Journal of the IMA
76 acute Scottish Hospitals; 1992/93	Number of staffed beds	Number of discharges and average length of stay for acute surgical/medical and other specialties Number of outpatient, accident and emergency and day case attendances	Scott and Parkin (1995), Health Economics

Other variables

In addition to the above identified input and output variables, a number of studies have identified an outstanding need to account for the differences between hospitals in terms of the services they provide, the patients they treat and their operational structures. Yong et al note that teaching hospitals typically perform more complex, innovative or rare procedures that may not be adequately reflected or captured by the inclusion of volume or casemix adjusted variables. They therefore include two dummy variables in their analysis to account for variation in costs across different types of hospitals in Victoria (25). Similarly, given expected differences in operating structures between metropolitan/large country and small rural hospitals in Victoria, the Steering Committee for the Review of Commonwealth/State Service Provision divides their sample of Victorian hospitals into the above two categories when making their assessment of efficiency (28).

A number of Canadian studies also distinguish between hospitals based on their size (32, 50, 52), geographic location (32, 50), teaching status (32, 52), and ownership type (government, religious, secular, not-for-profit) (50). Like Yong et al, Chowdhury et al make these distinctions between hospitals by inclusion of dummy variables in the DEA analysis (32), whilst Gruca et al utilise a nested DEA model to compare across hospital types (50) and Bilodeau et al utilise correlation coefficients (52). Given the similarities in geography and demography between Canada and Australia, it is unsurprising that variables relating to geographic location, size and teaching status were selected.

In their assessment of efficiency and productivity of English NHS Hospital Trusts, Castelli et al investigate variations in hospital productivity using ordinary least squares regressions (43). They utilise a number of explanatory variables identified in the literature as exerting an influence over performance at the hospital level, including variables such as the proportion of patients receiving some form of specialised care, teaching hospitals and Foundation Trust versus non-Foundation Trusts and the proportion of emergency admissions over total admissions (43). Similar approaches using regression analysis to investigate potential explanatory variables are also evident in a number of Canadian studies (31, 54, 55).

The Productivity Commission in their 2009 paper utilise a number of additional variables to account for differences in the nature and activities of hospitals across Australia, including the proportion of patients who are not treated as public patient as a proxy measure for the different levels of resources used by hospitals to treat public and non-public patient; and Evans and Walker information indices. These are measures of the relative complexity of work undertaken by a hospital, where the amount of information a hospital learns from an admission is inversely related to the likelihood of that case occurring within the system and that likelihood of that hospital treating that particular case (28, 61). Abeney and Yu utilise a regression analysis subsequent to their efficiency and productivity analysis to correct efficiency scores for the environmental and behavioural factors of patients, such as high blood pressure, asthma, post-secondary education, access to regular doctors, that may be present in a region’s patient population (53).

The inclusion of these types of additional variables is of particular importance, as it acknowledges that there are additional, indirect factors, such as the location and teaching status of the hospital that may influence a hospital’s efficiency and productivity. As teaching and research activity in Australia’s public hospitals increases (driven by initiatives such as the Medical Research Future Fund (62)), it will become increasingly important to capture the impact of this activity on overall hospital efficiency.

Approaches to productivity and efficiency analysis vary across Australia, Canada and the United Kingdom, but a number of common themes arise that can inform future studies of productivity and efficiency within Australia. These relate to national efforts to develop frameworks to assess health system efficiency; the selection of inputs, outputs and analysis technique; and the lack of integration and uptake of efficiency and productivity analysis into the health system.

Australia, the United Kingdom and to some degree Canada each have a national mechanism to assess the efficiency of public hospitals (12, 13, 21). However, these national systems do not provide true estimates of efficiency and therefore cannot be used to robustly identify areas of inefficiency across decision making units. As a result, a number of papers have called for improved frameworks and mechanisms to assess public hospital efficiency and productivity (20, 28) and there remains an outstanding need, particularly within Australia, for a comprehensive efficiency and productivity model that can be applied across the public hospital sector. Critical to any such model will be access to and availability of data, and the selection of appropriate inputs and outputs.

The selection of appropriate input and output variables is essential for robust measurement of public hospital efficiency and productivity. Input variables utilised by the studies identified in this review were similar, consisting of measures of labour (most commonly relating to full time equivalent employees), goods and services, e.g. purchased consumables, such as drugs, and capital. Treatment of capital and sourcing prices for material costs (goods and services) for Australian hospitals was often problematic and varied dependent upon individual hospital funding arrangements (25, 27, 28). In some instances the inclusion of capital was not considered appropriate for those public hospitals that do not own their own building assets, (25) was excluded from inclusion in analysis altogether, due to a lack of information available on capital assets (24, 29) or substituted for proxy measure such as number of beds (28). Similarly, studies excluded material costs altogether (25) or utilised nominal or total expenditures (28). In the United Kingdom, changes and discrepancies in health data and collection also impacted on the ability to select appropriate input (and output data), making comparisons across years difficult (23).

The selection of output variables is particularly important. Ideally, a hospital’s performance should be measured in terms of patient outcomes or other measures of quality of the care provided. Within Australia, however, there appears to be limited data available to directly measure patient outcomes or quality of care. As such, studies of Australian public hospitals have utilised so-called proxy measures of health performance that relate to hospital outputs, such as number of separations or occasions of service. Although such proxy measures are also utilised in studies of hospitals in the United Kingdom or Canada, some studies have utilised more direct measures of hospital performance, such as Potential Years of Life Lost (31) or median wait times (53). The United Kingdom productivity estimates, whilst not a true measure of efficiency, also apply quality adjustments, including factors such as measures of survival rate, estimated change in health outcomes following hospital treatment and mean life expectancy, change in inpatient waiting times (for elective, day case and non-elective activity and mental health care) and changes in outpatient waiting times. No Australian study utilised any such outputs relating to patient outcomes or quality of care and these studies provide a useful indication of how such approaches might be adopted in Australia, should such data become available.

In addition to the selection of input and output variables, many studies also investigated a number of other ‘environmental factors’ that may have contributed to identified inefficiencies. Commonly investigated characteristics included ‘type’ of hospital, e.g. teaching vs non-teaching hospitals, Trust vs non-Trust hospitals (UK) and geographic location. Such factors are highly relevant to Australia, where there is a significant variation in public hospital location and type. Of note, investigation of these factors was not restricted the type of analysis employed (e.g. DEA vs SFA).

Amongst the studies identified in this review, there was no clear consensus on a preferred analysis technique, whether this be DEA, SFA, Malmquist Indices or productivity indices (such as those employed by the United Kingdom). Within Australia SFA was employed most often, but this was not reflected elsewhere, with the majority of Canadian studies employing DEA and most recent United Kingdom studies employing a productivity index approach. With particular regard to DEA versus SFA, SFA when it was utilised was preferred for its ability to allow for measurement error, whilst DEA when it was employed for its greater flexibility, ability to handle multiple outputs and as a means to identify peers or role models by which benchmarks for performance can be established (29). A small number of studies employed a Malmquist Index approach for its ability to allow decomposition into efficiency and technology change (34) and the fact that it does not require input or output prices (47). What is clear from these studies is that the selection of analysis technique will be dependent upon the ultimate aims of the study and the availability of input and output data.

Perhaps the most striking observation from a reading of these identified efficiency and productivity analyses is the lack of engagement with the health system in the development and utilisation of these analyses. Those working within the health system are uniquely placed to advise on appropriate input and output variables that impact on efficiency and productivity in the sector, and most importantly, initiate appropriate action to drive improvements in efficiency and productivity. Yet there is little evidence of engagement with these stakeholders. Whilst there was some acknowledgement of the need for increased engagement with the health sector (19, 20) only one study actively consulted with health sector stakeholders in the development of their efficiency and productivity analysis (31). Moreover, no study reported any resultant uptake or impact by health system managers or policy makers. This lack of translation into the health system has previously been noted by Hollingsworth; of the close to 400 publications measuring economic efficiency in health care up to 2012, there has been little take-up of this evidence by policy makers (63).

In the face of rising costs of health expenditure across many countries, (64) there is an urgent need to consider the efficiency and productivity of the health system and in particular the public hospital sector, which is one of the largest sources of health expenditure within Australia. Although there a large number of studies globally that consider efficiency and productivity of the health system, (65) within Australia and key comparator nations, the number of efficiency and analysis studies is small. The number of studies that demonstrate active engagement with the health sector in study development and translation to impact is even smaller. These studies uniformly grapple with the selection of inputs and outputs that can accurately reflect the production function of a given hospital and the selection of a fit for purpose analysis technique. There is considerable scope for the further development of efficiency and productivity analysis techniques that can adequately capture relevant production factors, allow for robust comparisons across hospitals and time periods and which meaningfully engage with the health sector to inform improvements in efficiency and productivity.

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Availability of data and materials

Not applicable. Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

Competing interests

The authors declare that they have no competing interests.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author’s contributions

Bonnie Eklom: Conceptualization, Investigation, Writing – Original Draft Emily Callander: Writing – Review & Editing, Supervision

Acknowledgements

Not applicable

Schneider EC, Sarnak DO, Squires D, Shah A, Doty MM. Mirror, Mirror 2017: International comparison reflects flaws and opportunities for better U.S. health care. The Commonwealth Fund. 2017.
OECD. Health at a Glance 2017. Paris: OECD Publishing; 2017.
Department of Prime Minister and Cabinet. Reform of the federation white paper. Roles and responsibilities in health. Issues paper 3. Canberra: Australian Government; 2014.
National Commission of Audit. Towards responsible government. The report of the National Commission of Audit. Phase One. Canberra: Commonwealth of Australia; 2014.
Coelli TJ, Rao DSP, O'Donnell CJ, Battese GE. An introduction to efficiency and productivity analysis. 2 ed. United States of America: Springer US; 2005.
Farrell MJ. The Measurement of Productive Efficiency. Journal of the Royal Statistical Society Series A (General). 1957;120(3):253-90.
Auger CP. Information sources in grey literature. 4th ed. London; New Providence, N.J: Bowker-Saur; 1998.
Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Systematic reviews. 2015;4:138-.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ : British Medical Journal. 2015;349.
Pan B, Hembrooke H, Joachims T, Lorigo L, Gay G, Granka L. In Google We Trust: Users’ Decisions on Rank, Position, and Relevance. Journal of Computer-Mediated Communication. 2007;12(3):801-23.
Council of Australian Governments. National Health Reform Agreement. Canberra: Australian Government; 2011.
Australian Institute of Health and Welfare. About MyHospitals - MyHospitals overview | MyHospitals.gov.au: Australian Government; 2018https://www.myhospitals.gov.au/about-myhospitals/overview#performance-indicator-reporting. Accessed 20 May 2018.
Ariste R, Yu K. Comparisons of hospital output in Canada: national and international perspectives. International Journal of Health Economics and Management. 2017;17(4):433-51.
Sharpe A, Bradley C, Messinger H. The measurement of output and productivity in the health care sector in Canada: An overview. Report prepared for the Canadian Medical Association. Centre for the Study of Living Standards; 2007.
Crémieux PY, Ouellette P, Rimbaud F, Vigeant S. Hospital cost flexibility in the presence of many outputs: A public-private comparison. Health Care Management Science. 2005;8(2):111-20.
Crémieux PY, Ouellette P. Omitted variable bias and hospital costs. Journal of Health Economics. 2001;20(2):271-82.
Preyra C, Pink G. Scale and scope efficiencies through hospital consolidations. Journal of Health Economics. 2006;25(6):1049-68.
Yu K. Measuring efficiency and cost-effectiveness in the health care sector. In: Yu K, editor. Essays on the theory and practice of index numbers: the making of macroeconomics data. Verkag: VDM; 2011.
Lavis JN. Dialogue summary. Measuring health system efficiency in Canada. McMaster Health Forum; Hamilton, Canada 2011.
Canadian Institute for Health Information. Developing a model for measuring the efficiency of the health system in Canada, updated October 2012.
United Kingdom Parliament. NHS: Productivity: Written Question – 224314. 2015. https://www.parliament.uk/business/publications/written-questions-answers-statements/written-question/Commons/2015-02-11/224314/. Accessed 20 May 2018.
United Kingdom Office of National Statistics. Public service productivity estimates, healthcare: 2015. United Kingdom Office of National Statistics; 2018.
Castelli A, Chalkley M, Santana IR. Productivity of the English National Health Service : 2015/16 update. CHE research paper 152. Centre for Health Economics, York University; 2018.
Webster R, Kennedy S, Johnson L. Comparing techniques for measuring the efficiency and productivity of Australian private hospitals. Canberra: Australian Government; 1998.
Yong K, Harris A. Working Paper 92. Efficiency of hospitals in Victoria under casemix funding: A stochastic frontier approach. Melbourne: Centre for Health Program Evaluation; 1999.
Wang J, Zhao Z, Mahmood A. Relative efficiency, scale effect and scope effect of public hospitals: Evidence from Australia. IZA Discussion Paper Series. 2006;IZA DP No. 2520.
Gabbitas O, Jeffs C. Assessing productivity in the delivery of public hospital services in Australia: some experimental estimates. 30th Australian Health Economics Conference; 2-3 October 2008; Adelaide2008.
Productivity Commission. Public and Private Hospitals. Canberra: Australian Government; 2009.
Steering Committee for the Review of Commonwealth/State Service Provision. Data Envelopment Analysis: A technique for measuring the efficiency of government service delivery. Canberra: Australian Government; 1997.
Fixler T, Paradi JC, Yang X. A data envelopment analysis approach for measuring the efficiency of Canadian acute care hospitals. Health Services Management Research. 2014;27(3-4):57-69.
Allin S, Grignon M, Wang L. The determinants of efficiency in the Canadian health care system. Health Economics Policy and Law. 2016;11(1):39-65.
Chowdhury H, Zelenyuk V. Performance of hospital services in Ontario: DEA with truncated regression approach. Omega (United Kingdom). 2016;63:111-22.
Asmild M, Hollingsworth B, Birch S. The scale of hospital production in different settings: one size does not fit all. Journal of Productivity Analysis. 2013;40(2):197-206.
Chowdhury H, Zelenyuk V, Laporte A, Wodchis W. Analysis of productivity, efficiency and technological changes in hospital services in Ontario: How does case-mix matter? International Journal of Production Economics. 2014;150:74–82.
Chowdhury H, Wodchis W, Laporte A. Efficiency and technological change in health care services in Ontario: An application of Malmquist Productivity Index with bootstrapping. International Journal of Productivity and Performance Management. 2011;60(7):721-45.
Asmild M, Tam F. Estimating global frontier shifts and global Malmquist indices. Journal of Productivity Analysis. 2007;27(2):137-48.
Hollingsworth B, Parkin D. Efficiency and productivity change in the English National Health Service: Can data envelopment analysis provide a robust and useful measure? Journal of Health Services Research and Policy. 2003;8(4):230-6.
Bevan G, Karanikolos M, Exley J, Nolte E, Connolly S, Mays N. The four health systems of the United Kingdom: how do they compare? : London School of Hygiene & Tropical Medicine; 2014.
Street A, Jacobs R. Relative performance evaluation of the English acute hospital sector. Applied Economics. 2002;34(9):1109-19.
Dawson D, Goddard M, Street A. Improving performance in public hospitals: A role for comparative costs? Health Policy. 2001;57(3):235-48.
Scott A, Parkin D. Investigating hospital efficiency in the new NHS: The role of the translog cost function. Health Economics. 1995;4(6):467-78.
Street A. How much confidence should we place in efficiency estimates? Health Economics. 2003;12(11):895-907.
Castelli A, Street A, Verzulli R, Ward P. Examining variations in hospital productivity in the English NHS. The European Journal of Health Economics. 2015;16(3):243-54.
Castelli A, Laudicella M, Street A, Ward P. Getting out what we put in: productivity of the English National Health Service. Health Economics, Policy and Law. 2011;6(3):313-35.
Bojke C, Castelli A, Grašič K, Street A. Productivity Growth in the English National Health Service from 1998/1999 to 2013/2014. Health Economics. 2017;26(5):547-65.
Aragon MJA, Castelli A, Gaughan J. Hospital Trusts productivity in the English NHS: Uncovering possible drivers of productivity variations. PLoS One. 2017;12(8):e0182253.
Valdmanis V, Rosko M, Mancuso P, Tavakoli M, Farrar S. Measuring performance change in Scottish hospitals: a Malmquist and times-series approach. Health Services and Outcomes Research Methodology. 2017;17(2):113-26.
Productivity Commission. Public and private hospitals: Multivariate analysis. Canberra: Australian Government; 2010.
Braithwaite J, Westbrook MT, Hindle D, Iedema RA, Black DA. Does restructuring hospitals result in greater efficiency - an empirical test using diachronic data. Health Services Management Research. 2006;19(1):1-12.
Gruca TS, Nath D. The technical efficiency of hospitals under a single payer system: The case of Ontario community hospitals. Health Care Management Science. 2001;4(2):91-101.
Bilodeau D, Crémieux P-Y, Jaumard B, Ouellette P, Vovor T. Measuring Hospital Performance in the Presence of Quasi-Fixed Inputs: An Analysis of Québec Hospitals. Journal of Productivity Analysis. 2004;21(2):183-99.
Bilodeau D, Crémieux P-Y, Ouellette P. Hospital performance in a noncompetitive environment. Applied Economics. 2009;41(4):459-68.
Abeney A, Yu K. Measuring the efficiency of the Canadian health care system. Canada: Lakehead University; 2015.
Allin S, Veillard J, Wang L, Grignon M. How can health system efficiency be improved in Canada? Healthcare Policy. 2015;11(1):33-45.
Canadian Institute for Health Information. Health system efficiency in Canada: Why does efficiency vary among regions? Canada; 2014.
Duckett SJ. Hospital payment arrangements to encourage efficiency: the case of Victoria, Australia. Health Policy. 1995;34(2):113-34.
University of British Columbia Centre for Health Services and Policy Research. Current hospital funding in Canada. The limitations of global budgets.; 2014.
Canadian Institute for Health Information. Case Mix | CIHI. 2018. https://www.cihi.ca/en/submit-data-and-view-standards/methodologies-and-decision-support-tools/case-mix. Accessed 30 May 2020.
The Commonwealth Fund. England : International Health Care System Profiles. 2018. https://international.commonwealthfund.org/countries/england. Accessed 20 June 2020.
Allin S, Grignon M, Wang L. The determinants of efficiency in the Canadian health care system. Health Economics, Policy and Law. 2014;11(1):39-65.
Evans RG, Walker HD. Information theory and the analysis of hospital cost structure. The Canadian Journal of Economics. 1972;5(3):398-418.
Australian Medical Research Advisory Board. Medical Research Future Fund: Australian medical research and innovation strategy 2016-2021. Canberra: Australian Government; 2016.
Hollingsworth B. Revolution, evolution, or status quo? Guidelines for efficiency measurement in health care. Journal of Productivity Analysis. 2012;37(1):1-5.
Marino A, Morgan D, Lorenzoni L, James C. Future trends in health care expenditure. 2017.
Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health Economics. 2008;17(10):1107-28.

AdditionalFile1PRISMA2009checklistrevised07122020.doc

Download PDF

Version 1

posted

You are reading this latest preprint version

A Systematic Review of Hospital Efficiency and Productivity Studies: Lessons from Australia, UK and Canada.

Status:

Version 1

Abstract

Figures

Background

Methods

Search strategy and selection criteria

Selection criteria

(1) Grey literature

(2) Academic databases search

Data extraction

(1) Academic databases

(2) Grey literature

Quality assessment

Data analysis

Results

National measures of hospital performance

Analysis Technique

Outputs

Inputs

Other variables

Discussion

Conclusion

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing interests

Funding

Author’s contributions

Acknowledgements

References

Supplementary Files

Status:

Version 1