We used data from the retrospective longitudinal analysis of the project entitled “Cost-effectiveness analysis of the UNEMES-EC model for multidisciplinary care of patients with type 2 diabetes mellitus” (Sosa-Rubí et al. 2020). Random sampling was carried out to select 40 units: 20 units that provide traditional healthcare (doctor-nurse), and 20 units that offer multidisciplinary healthcare services (doctor, nurse, psychologist, nutritionist and physical trainer).
Our analysis plan was informed by previous related work (Sosa-Rubí et al. 2021); and we conducted our analysis in 4 stages. First, we estimated a technical efficiency score for THC and MHC in relation to patient care using data envelopment analysis (Ramanathan 2003). Second, we calculated the quality score (performance and competence) for each type of model. Third, we mapped the relationship between the efficiency and quality scores, and we studied the managerial determinants of best performance in terms of both efficiency and quality using a positive deviance regression approach (Pascale et al. 2010).
Main Variables
Technical efficiency
We estimated the technical efficiency of the different healthcare models using data envelopment analysis (DEA). DEA is a nonparametric linear programming technique used to measure technical efficiency in a sample of homogeneous decision-making units such as health facilities (Charnes et al. 1978). In such cases, DEA measures the extent to which a healthcare unit achieves a given level of multi-dimensional output relative to its consumption of multi-dimensional inputs (Sherman and Zhu, 2006). Specifically, DEA identifies the most efficient units, and it pinpoints those susceptible to improvement. We calculated efficiency scores in percentage terms, defined as the distance between the efficient and the susceptible units, where 100% means that within the sample, the unit is the most effective; and 0% means that the unit is the least effective (or most susceptible). Therefore, the scores provided an efficiency ranking of healthcare units (Kirigia and Asbu 2013).
We interpreted efficiency as the ratio of outputs to inputs. In a healthcare context, these outputs are frequently measured as a count of patients treated or procedures performed, while inputs commonly refer to money expenditures or staff full-time equivalents (McGlynn and Shekelle 2008). As suggested in the literature, we considered the total number of T2D patients as an output variable (Jacobs et al. 2006). We collected data on costs of medications, staff, equipment, general services and training from previous work, as inputs (Sosa-Rubí et al. 2020).
We implemented output-oriented variable-returns-to-scale DEA models (Banker et al. 1984). Thus, the resulting efficiency scores represent the proportional increases in outputs (number of patients with T2D) that each healthcare unit could achieve using the same level of inputs (at the same costs) if it were at the frontier of efficient production. The model was as follows:
$${\theta }_{n}=\text{max}\sum _{j=1}^{q}{u}_{jn} {y}_{jn }- {w}_{n}, \forall n ϵ \left\{\text{1,2}, \dots , k\right\} \left(1\right)$$
$$s.t. \sum _{i=1}^{r}{v}_{in}{x}_{in}\le 1$$
$$\sum _{j=1}^{q}{u}_{jn}{y}_{jn}- {w}_{n}- \sum _{i=1}^{r}{v}_{in}{x}_{in} \ge 0, \forall n ϵ \left\{\text{1,2}, \dots , k\right\}$$
$${u}_{jn}\ge 0, {v}_{in }\ge 0 \forall i ϵ \left\{\text{1,2}, \dots , r\right\}; \forall j ϵ \left\{\text{1,2}, \dots , q\right\}.$$
$${w}_{n}\mathbb{ }\in \mathbb{ }\mathbb{R}$$
where \({\theta }_{n}\) is the technical efficiency score of the nth unit; \({u}_{jn}\) and \({v}_{in}\) are the relative weights of the ith input and the jth product of the nth unit. The values of outputs (yjn) and inputs (xin) are constrained to be positive or equal to zero; and \(u and v\)are positive vectors. The sum of inputs is normalized to the unit. Lastly, the term \({w}_{n}\) indicated the returns to scale. We found efficiency scores that identified the best practice frontier by solving the optimization problem presented in Eq. (1). We used bootstrapping due to the deterministic nature of the DEA method, and the fact that the distribution of the estimators of interest was unknown (Banker, 1996; Simar and Wilson, 2007). The resampling was generated using an estimate of the data generating process (Löthgren and Tambour, 1996). We used R software for all DEA analyses (Bogetoft and Otto 2011).
Quality of care
We computed two indices to measure quality of care at the facility level – both of which were measures of process quality (Das and Gertler 2007). First, to measure the physicians’ ability to manage patient care, we calculated a provider competence index based on responses to vignette instruments (Das and Hammer 2005). We used this method to assess the extent to which providers followed existing national guidelines to screening and treating diabetes patients. We selected providers for each different healthcare model. The vignettes presented respondents with hypothetical scenarios describing typical diabetes patients. Each respondent was then asked a series of questions on how they would deal with the clients in these hypothetical scenarios. This method allowed us to measure the gap between the procedures that should be followed, according to the national guidelines, and the procedures that the medical staff mentioned they would perform in a hypothetical case (see details about these vignettes in Appendix Table A1).
Second, and following previous studies (Rannan-Eliya et al. 2015) we estimated a provider performance index based on responses to patient exit interviews. The exit interview module was directed at randomly selected diabetes patients at each facility. The exit interviews were designed to collect information on the process quality of the visit from the perspective of the patient, following the same structure and including the same components as the provider vignettes (see details about these exit interviews in Appendix Table A2). This quality indicator allowed us to measure the gap between all medical procedures that patients should have received during their visit to the health facility, based on what the national guidelines recommend, and what the patients actually received during their visit (Hutchinson et al. 2011). We used principal component analysis (Rencher 2003) to construct provider competence and provider performance indices from the responses to the provider vignettes and patient exit interviews (Jolliffe 2002). We retained the first principal component of each measure and rescaled it to be bounded between zero and one hundred. We confirmed the viability and relevance of the index through different statistical tests (Appendix Table A3). For all quality analyses we used STATA software version 15 (StataCorp 2017).
Best Performance
Based on the technical efficiency and quality of care metrics, we graphically displayed the bias corrected efficiency scores against our two quality scores (performance and competence). We identified the healthcare units with the best performance in diabetes care as those located in the upper right-hand quadrants: High Efficiency and High Performance (HE & HP), and the High Efficiency and High Competence (HE & HC). In each analysis, we defined “high performance” as performance above the median for all healthcare facilities.
Management Practice Covariates
We derived the management variables from responses to questions on facility-level management practices. The managerial processes were used as predictors of the differences between the healthcare models. Indirect predictors of the quality of care were measured through proxy questions on key dimensions, based on the literature (Bradley et al., 2015; Sosa-Rubí et al., 2021). The management variables were dichotomous, including: (a) existence of evening shift; (b) state supervision of patient care; (c) modifications to treatment due to shortage of medicines; (d) learning management practices through courses by the state; (e) medical license review; (f) state supervision of diabetes-related services; (g) rotation by services or offices among staff; and (h) staff performance evaluation. (Appendix table A4 and table A5).
Analysis
We identified management characteristics associated with an increase in the probability (Y) of the nth health facility being classified in the high quadrants described above, (HE & HP) or (HE & HC), as follows:
$$Pr\left({Y}_{n}=y|{X}_{n}\right)={X}_{n}\beta +{\epsilon }_{n} \left(4\right)$$
where \({X}_{n}\) includes binary and count variables that characterize management practices described above.