Effectiveness and efficiency of leadership, management, and governance trainings in Ethiopia: a cross-sectional study

doi:10.21203/rs.3.rs-672515/v1

Download PDF

Research note

Effectiveness and efficiency of leadership, management, and governance trainings in Ethiopia: a cross-sectional study

https://doi.org/10.21203/rs.3.rs-672515/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Objectives

Leadership, management, and governance (LMG) trainings are implemented to capacitate managers and clinical healthcare providers in several countries. However, there is no evidence gathered on effectiveness and efficiency of LMG trainings in low-income countries. Hence, this study aims to determine the effectiveness and efficiency of basic LMG trainings implemented in Ethiopia.

Results

In the study, the results training evaluations of 293 health workers with a 100.0% response rate is presented. In addition, data on 136 leadership project achievements and financial documents are reviewed. The mean pretest score with SD was 44.97% ± 15.50%, and the post-test score was 78.11% ± 14.22%, which showed a significant gains in knowledge at post-test with t=-35.99, df = 292, p < 0.001. The average net health service coverage gain by each entity was 16.55% (95% CI: 12.26%, 20.84%). The majority of LMG trainings were found to be effective in imparting knowledge and skills for staff to lead, manage and govern primary health entities. Moreover, the trainings were efficient in regards to time, communication, and resources utilized during the sessions. The trainings had a synergetic effect in strengthening health system components in the low-income country of Ethiopia.

Leadership and Ethics

Medical Ethics

Effectiveness and Efficiency

Leadership

Management

and Governance

Performance Improvement

Training Evaluation

Ethiopia

Training is the act of enriching the knowledge and skills of an employee in a particular job, to improve organizational performance and healthcare effectiveness [1] [2]. LMG training is a results-oriented, participatory leadership development program that enables healthcare professionals who play a hybrid of clinical and management roles to overcome self-identified challenges and achieve more results. Following the trainings, the trainees are expected to apply all leading, managing, and governing practices [2].

Effectiveness is focused on the usefulness of training contents to the employees’ work. Efficiency is the (often measurable) ability to avoid wasting materials, energy, efforts, money, and time in doing something or producing a desired result [3] [4]. Conducting LMG training effectiveness and efficiency evaluation is helpful for policy makers, program managers and health professionals to maximize the gains of such interventions and improve training materials and processes. The objectives of this study are to uncover the time, communication and resources utilized throughout LMG trainings, and investigate the effectiveness and efficiency of LMG trainings being implemented as a performance improvement tool at primary healthcare entities in Ethiopia.

Operational definitions

Time: trainees were asked to rate the adequacy of time spent on each training topic/session and activity.

Communication: trainees were asked to evaluate the various levels of communication, including the invitation to participate in the training, the purpose of the training, and post-training communication with trainers and program managers.

Resources: trainees were asked to evaluate training materials, tools, and any additional materials used to assist trainees during the training sessions.

Study area and design

This study employed a facility based cross-sectional survey design and was conducted in September 2018. The study area was purposively selected, targeting LMG training implementor districts in four agrarian regions of Ethiopia. LMG trainings were facilitated to capacitate the knowledge, skills, and competencies of the health workforce. In 2017, 24 districts with 555 participants were covered with LMG trainings [5].

Sample size determination and sampling procedure

The sample size was calculated using a single population formula [6]. The assumptions followed were: the prevalence of effectiveness of LMG trainings (P) is 50% (where, p = 0.50, q = 0.50), allowing 5% for expected margin of error (d), 95% confidence level (Z α/2 = 1.96), 25% for non- response rate, and 555 LMG trained staff (population). The total enrolled participants in the study were 293. The determined sample size was proportionally allocated to each intervention district based on the number of trainees. All invited health workers participated in the study.

Data collection and analysis

The first tool used in the study consists of 30 measurement items/questions. Of these, 19 were dedicated to capturing the independent variables (i.e. time, communication, and resources); and 11 were focused on effectiveness and efficiency of training events that are the dependent variables of this study. The tools were developed based on a 5-point Likert scale [7] ranging from strongly disagree to strongly agree. The scale values are 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree. In addition, data extraction tools were developed and used to capture theoretical exam and leadership project scores, and financial data related to LMG trainings, coaching and knowledge sharing events.

The data were analyzed using SPSS IBM version 20 [8]. The tools’ internal reliability was assessed using Cronbach’s alpha values. According to Bland and Altman (1997), if the Cronbach alpha value score is more than 0.7, the scale can be considered reliable [9]. The Cronbach’s alpha values in this study were 0.830, 0.823, 0.730, 0.824, and 0.806 for time, resources, communication, effectiveness, and efficiency, respectively. The results showed that the tools used were internally consistent and reliable.

Descriptive statistics was employed, and the results were presented using frequency tables, graphs, and mean and standard deviations. A paired sample t-test was analyzed to check for differences in cognition and practice. In addition, linear regression was used to test the relationship among each dependent and independent variable. Multivariable linear regression analysis was used to identify the predictors of LMG training effectiveness and efficiency.

Socio-demographic characteristics of participants

One-hundred and three trainees, 35.1% from SNNP, 28.0% from Oromia, 26.3% from Amhara, and 10.6% from Tigray regions were enrolled. The majority, 75.8% of participants were male. The mean age of participants with standard deviation (SD) was 29.9 ± 6.9 years. Among LMG trainees, 60.8% had attended the segmented LMG approach training course, while the rest, 39.2% had attended the block LMG training approach training course.

LMG trainee evaluations

Figure1 illustrates the results of the LMG trainee evaluations using three independent and two dependent variables. The lowest average score was 3.83 ± 0.80 for time assessment variables. While the highest average score was 4.30 ± 0.59 for effectiveness assessment variables (additional file 1).

Effectiveness and efficiency of LMG trainings

The mean pretest score with SD was 44.97% ± 15.50%. The post-test score was 78.11% ± 14.22%, which showed a significant gain in knowledge at post-test with t=-35.99, df = 292, p < 0.001. In this study, 80.9% (237) of trainees scored greater or equal to 70% on post-tests. All 136 leadership projects of the LMG trainees were assessed for which the mean baseline score with SD was 38.35 ± 24.79 units and mean achieved result with SD was 52.22 ± 26.23 units. The majority, 252 (86%) of trainees achieved 80% or more on their leadership projects. As a result of these projects, 2,290.6 units of desired measurable results were gained. The average net gain by each entity was 16.55 (95% CI: 12.26, 20.84) with SD (± 17.59) units. The result gained, with paired sample t-test t= -12.28, df = 292, p < 0.001, was statistically significant.

The result of linear regression analysis revealed that time is explained by 6.6% of variability in effectiveness and 17.5% of efficiency. Similarly, communication is explained by 17.9% of variance in effectiveness, and 27.4% of the variance in efficiency. Moreover, resources explain 33.4% of variance in effectiveness and 32.3% of variance in efficiency. Table 1 presents the effects of time, communication, and resources on effectiveness and efficiency of LMG trainings. The beta (β) coefficient from the general linear models for LMG training effectiveness, unadjusted score with 95% confidence interval (CI) for communication was β 0.127(0.048, 0.175), and resources was β 0.473 (0.419, 0.527). The β coefficient for LMG training efficiency, unadjusted score with 95% CI for resources was β 0.351(0.298, 0.404), communication was β 0.214(0.166, 0.262), and time was β 0.123 (0.084, 0.162). The relationship was found statistically significant at p < 0.05. The following (1) & (2) final models of effectiveness and efficiency were developed.

Table 1

Linear regression coefficients, September 2018
Model		Unstandardized coefficients		Standardized coefficients	t	Sig.
Model		B	Std. error	Beta	t	Sig.
Effectiveness (1)	(Constant)	1.848	.206		8.982	0.001
	Time	− .003	.040	− .003	− .064	0.949
	Communication	.128	.050	.152	2.554	0.011
	Resources	.474	.056	.497	8.537	0.000
Effectiveness (2)	(Constant)	1.845	.200		9.247	0.000
	Communication	.127	.048	.151	2.652	0.008
	Resources	.473	.054	.496	8.733	0.001
Efficiency (3)	(Constant)	2.010	.193		10.404	0.001
Efficiency (3)	Resources	.549	.046	.571	11.857	0.001
Efficiency (4)	(Constant)	1.656	.195		8.478	0.001
	Resources	.387	.053	.403	7.297	0.001
	Communication	.258	.047	.302	5.480	0.001
Efficiency (5)	(Constant)	1.507	.198		7.610	0.001
	Resources	.351	.053	.366	6.578	0.001
	Communication	.214	.048	.251	4.417	0.001
	Time	.123	.039	.164	3.173	0.002

Effectiveness = 1.845 + 0.127*, Communication + 0.473*, Resources (1)
Efficiency = 1.507 + 0.351*, Resources + 0.214*, Communication + 0.123* Time (2)

Return on investments

Of the 136 projects implemented, 52 (38.5%) were dedicated to improving skilled delivery services; 20 (14.8%) on antenatal care services; 25 (18.5%) on family planning (FP) services; and 25 (18.5%) on strengthening health systems. The projects invested $52,719.99 to gain an additional 2,290.6 health services coverages. The results indicate that to gain one unit of skilled delivery service coverage, the projects invested $32.92 on LMG training and interventions; similarly, to improve one unit of FP coverage, the project invested $4.95. These investments varied between regions. The least cost-per-unit of gained health service coverage of $18.72, was recorded in SNNP region, while the highest cost-per-unit of gained health service coverage of $31.57, was recorded in Oromia region. Furthermore, considering the training approaches as an independent variable, the least investment-per-unit-gained health service coverage was $19.20 for block a basic LMG training approach; the highest recorded for a segmented II training approach was at $34.6 (Table 2).

Table 2

Return on investment of LMG trainings, September 2018
Ser no.	Theme	Number of projects	Gains in coverages	Total cost in USD	Unit cost /gained coverages (ETB)	Unit cost /gained coverages (USD)
1	Program area
1.1	Delivery: maternal and neonatal health	53	714.7	23,524.70	905.18	32.92
1.2	Antenatal care: maternal and neonatal health	20	372.6	60,38.37	445.67	16.21
1.3	Postnatal care: maternal and neonatal health	2	24	358.71	411.02	14.95
1.4	Family planning	25	466.6	9,649.89	568.74	20.68
1.5	Health systems	25	355.7	8,856.66	684.73	24.90
1.6	Child health and immunization	5	97	1,486.83	421.52	15.33
1.7	Community-based health insurance	4	178	1,681.87	259.84	9.45
1.8	Other (malaria and tuberculosis prevention/control)	2	82	1,122.96	166.93	13.69
1.9	Overall leadership projects	136	2,290.6	52,719.99	632.93	23.02
2	Regions
2.1	Amhara	37	607.3	13,480.22	610.42	22.20
2.2	Oromia	34	564.9	17,836.56	868.30	31.57
2.3	SNNP	43	830.0	15,541.05	514.91	18.72
2.4	Tigray	21	288.4	5,862.15	558.98	20.33
3	Training approaches
3.1	6-day block basic LMG training	58	1,045.3	20,074.00	528.11	19.20
3.2	Segmented I: two 3-day basic LMG trainings	60	1,062.7	26,328.14	681.31	24.77
3.3	Segmented II: two 3-day basic LMG trainings	17	182.6	6,317.85	951.48	34.60

In 2007, the world health organization (WHO) recommended a framework for strengthening health systems and improving health outcomes through synergetic interventions [10] [11]. The USAID Transform: Primary Health Care project has supported the implementation of LMG trainings (intervention) at the primary health level in 24 districts. This study documented the results of 293 LMG trainees who developed and implemented 136 leadership projects in Ethiopia [5]. The evidence generated on effectiveness and efficiency of the LMG training evaluations enable policy makers, program managers, and health professionals to decide on training approaches and contents based on evidences.

In this study, the effectiveness of the basic LMG trainings was assessed by measuring the knowledge, skills, and behavioral patterns of trainees within primary healthcare facilities [3] [4]. The majority of trainees successfully completed the LMG trainings. In addition, the evaluation indicated that the contents of the LMG training helped to improve primary healthcare service coverage over 12 months. This result was also attributed to the implementation of multi-faceted interventions, which ranged from classroom trainings to leadership-project implementation, and onsite coaching by experts [12]–[15]. The findings also concur with La Rue et al. (2012), who compared leadership exposed and non-exposed groups and revealed higher and statistically significant differences in health service coverages in the first group [16]. Our findings also concurred with Rowe et al.’s (2018) systematic review, which reported that professional healthcare provider practices were more likely to improve using multiple strategies than by a single strategy [17].

Furthermore, effectiveness and efficiency of LMG trainings were considered as dependent variables. The dependent variables were computed against time, communication, and resources dedicated for the project, trainers, and the public health system as independent factors. The results of this study revealed that resources and communication had a statistical significantly positive impact on the effectiveness of the LMG trainings while being directly related with efficiency of the LMG training. Moreover, an increase of health services coverage by one unit or percent required an investment of 23.02 USD from the project. This study indicates that block basic LMG trainings were more effective and efficient than the segmented training approaches. The results showed that participants in the block approach were more effective and efficient than segmented LMG trainees.

Leadership, management, and governance training was found to be effective in imparting knowledge and skills on leading, managing, and governing practices in primary healthcare entities and improving organizational performances. LMG trainings were found to be efficient in time, communication, and resource utilization during facilitation. The LMG trainings had a synergetic effect in strengthening the health system in the low-income country of Ethiopia.

Like many cross-sectional studies, this study has limitations to claim causal relationships. Since the data collection was made after completing the training, there might be recall bias. The study was conducted only in USAID Transform: Primary Health Care project targeted sites, hence, before generalizing the findings, the results should be interpreted based on the context.

LMG: Leadership, Management and Governance; SD: Standard Deviation; SNNP: southern Nations, Nationalities and Peoples’; USAID: United States Agency for International Development; WHO: World Health Organization

Ethics approval and consent to participate

Ethical clearances were granted by the Amhara Public Health Institute (Ref No. HRTT02/137/2018), the Oromia Regional State Health Bureau (Ref No. BEFO/HBTPH/1-8/476), the SNNP Regional State Health Bureau (Ref No. PLMG-19/8407), and the Tigray Regional State Health Bureau’s (Ref No. 453/1418/10) institution review boards and research ethics committees. Data were collected after getting full informed written consent from each participant and facility manager. Privacy, anonymity, and confidentiality were maintained throughout the data collection, analysis, and report writing activities. The study had no known risk and no payment was made to participants.

Availability of data materials

The datasets used and/or analyzed during this study are available from the corresponding author upon reasonable request.

Funding

This study was fully funded by USAID under cooperative agreement AID-663-A-17-00002 as part of the USAID Transform: Primary Health Care Activity in Ethiopia. The funding body had no role in the design, data collection, analysis, interpretation and writing stages of the study undertaking.

Acknowledgements

The investigators are indebted to all participants for their cooperation during data collection.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

Conceived and designed the experiments: SAM, BFD, WSA, HD, MDA.

Performed the experiments: SAM, BFD, WSA, HD, MDA.

Analyzed the data: BFD, MDA.

Contributed reagents/materials/ analysis tools: SAM, BFD, WSA, HD, MDA.

Wrote the paper: BFD, MDA, HD.

All authors have read and approved the final manuscript.

Author details

¹USAID Transform: Primary Health Care, JSI Research & Training Institute, Inc., Addis Ababa, Ethiopia.

Armstrong M. Armstrong's handbook of performance management: An evidence-based guide to delivering high performance. 3: Kogan Page Publishers; 2009 Sep.
Federal Ministry of Health (FMOH). Leadership, Management and. In: Governance In-Service Training Manual for Health Managers at Hospitals and Health Centers. FMOH: Addis Ababa; 2017.
Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick's four levels of training evaluation. Association for Talent Development; 2016 Oct. p. 1.
Tetui M. Participatory approaches to strengthening district health managers' capacity: Ugandan and global experiences (Doctoral dissertation, Umeå universitet); 2018.
USAID Transform. Primary Health Care Project. Theory of change in practice 2017. USAID TPHC: Addis Ababa. 2017.
Pagano M, Gauvereau K. Principles of Biostatics. 2nd Edition, Duxbury, Pacific Grove. 2000.
Likert R. A technique for the measurement of attitudes. Archives of psychology. 1932.
SPSS I. IBM SPSS statistics base 20. Chicago, IL: SPSS Inc. 2011;15.
Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BmJ. 1997;314:572. https://doi.org/10.1136/bmj.314.7080.572.
World Health Organization. Everybody's business–strengthening health systems to improve health outcomes: WHO's framework for action. World Health Organization; 2007.
Egger D, Travis P, Dovlo D, Hawken L, World Health Organization. Strengthening management in low-income countries. World Health Organization; 2005.
Argaw MD, Desta BF, Muktar SA, Tewfik N, Tefera BB, Abera WS, Bele TA, Buseir SS, Rogers D, Eifler K. Effectiveness of Leadership, Management, and Governance Competencies for Performance Improvements at Primary Health Care Entities in Ethiopia: A Before and After Study. Journal of Human Resource and Sustainability Studies. 2021 Apr 23;9(2):250 – 75.
Desta BF, Abitew A, Beshir IA, Argaw MD, Abdlkader S. Leadership, governance, and management for improving district capacity and performance: the case of USAID transform: primary health care. BMC Fam Pract. 2020 Dec;21(1):1–7.
Mutale W, Vardoy-Mutale AT, Kachemba A, Mukendi R, Clarke K, Mulenga D. Leadership and management training as a catalyst to health system strengthening in low-income settings: Evidence from implementation of the Zambia Management and Leadership course for district health managers in Zambia. PLoS One. 2017 Jul 25;12(7):e0174536.
Uneke CJ, Ezeoha AE, Ndukwe CD, Oyibo PG, Onwe FD. Enhancing leadership and governance competencies to strengthen health systems in Nigeria: assessment of organizational human resources development. Healthcare Policy. 2012 Feb;7(3):73.
La Rue KS, Alegre JC, Murei L, Bragar J, Thatte N, Kibunga P, Cheburet S. Strengthening management and leadership practices to increase health-service delivery in Kenya: an evidence-based approach. Human resources for health. 2012 Dec;10(1):1–7.
Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. (2018). Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. The Lancet Global Health 1;6(11):e1163-75.

Additionalfile1.docx
Evaluation of LMG training by participants, September 2018 The table presents the details rating of trainees on time communication, resources, effectiveness, and efficiency.

Download PDF

Version 1

posted

You are reading this latest preprint version

Effectiveness and efficiency of leadership, management, and governance trainings in Ethiopia: a cross-sectional study

Status:

Version 1

Abstract

Objectives

Results

Figures

Introduction

Operational definitions

Materials And Methods

Study area and design

Sample size determination and sampling procedure

Data collection and analysis

Results

Socio-demographic characteristics of participants

LMG trainee evaluations

Effectiveness and efficiency of LMG trainings

Return on investments

Discussion

Conclusions

Limitations Of The Study

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1