Review of the literature revealed 26 organizational readiness tools related to health institutions (including hospitals, clinics, research institutes, etc.) and 30 tools developed specifically to assess organizational readiness for implementing global health interventions. Both sets of tools had been collated and analyzed in systematic reviews (23, 26). Common constructs across these tools included availability of resources, individual attributes and motivation, and organizational climate.
Eighteen stakeholders, both internal and external to the 6 STRIPE academic institution were interviewed. Three cross-cutting themes emerged as relevant for readiness to conduct KT in LMICs. This included 1) the complexity of the policy process and necessity of “soft-skills”, 2) misalignment between institutional missions and incentives and 3) the role of internal and external networks. The results from the consultation process are described in more detail elsewhere (9). The constructs identified from the literature review and the cross-cutting themes from the consultation process were organized into a quantitative tool with 5 domains and 76 items, with new items developed for the themes. A total of 9 additional questions on demographics, facilitators, and barriers to doing KT were added to translate the tool into a survey questionnaire [Appendix 1].
We received 158 responses to the survey across the 6 institutions and TWG. There were 47 respondents who completed 9% or less of the survey, which were subsequently dropped from analysis. These respondents did not cluster by country, age, or gender and appeared random. A total of 111 responses were included in the final analysis. According to Arrindell and van der Ende, a sample size (N) to items (p) ratio, i.e. N:p ratio of 3:1 is adequate for demonstrating a stable factor structure with an alpha level of 0.05 (32). Hence, our sample size of 111 will be adequate for demonstrating the validity and reliability of a tool with at least 37 items.
There were 27 respondents from Bangladesh, 19 from Indonesia, 16 from India, 16 from Nigeria, 12 from DRC, 11 from Ethiopia, and 10 from the TWG representing other LMICs. Of the total 111 respondents, 53% (n=59) of respondents were male and 43% (n=48) were female. A majority of respondents (57%, n=64) were 30-49 years of age and most (59%, n=66) indicated they had experience conducting KT. Participants were asked to indicate their current professional focus/foci. Most respondents were engaged in research (83%, n=93) and teaching (62%, n=69). Other common foci included project coordination (36%, n=41), leadership (15%, n=17), and management (16%, n=18).
The most commonly conducted KT activities included “taught a course on communication, advocacy, stakeholder engagement or KT” (98%, n=109), “conducted a stakeholder meeting” (62%, n=69), and “given a presentation at a scientific conference” (60%, n=67, ). Individuals who indicated having experience with KT activities were significantly more likely to have written a policy brief (p-value 0.0059), conducted a stakeholder meeting (p-value 0.0364), engaged with policy makers to set priorities (p-value 0.0129), and to have given a presentation at a scientific conference (p-value 0.0011). Two KT activities also varied significantly by country, “authored or co-authored an article in a peer-review journal” (p-value 0.0001), and “given a presentation at a scientific conference” (p-value 0.0002). Additional descriptive statistics are presented in Table 2.
[Insert Table 2]
We ran an exploratory factor analysis on the complete data set (version 0) which included 76 items. This approach yielded 22 factors; both the KMO measure and Barlett’s test yielded no value. Constructs covered in these factors included individual motivation, organizational climate, organizational culture, internal resources, individual knowledge and skills, internal and external networks, funding sources, prioritization, and shared ethos for change (change valence). Many factors overlapped, each addressing similar or related constructs. The correlation matrix was reviewed for highly correlated items and those with correlations above 0.5 were dropped; 17 items were removed in this process. Highly correlated items included, “Q1: I am confident that I can conduct KT activities”, “Q3: I feel personally motivated to do KT”, and “Q10 I have the skills to conduct KT”. Q10 was also heavily correlated with “Q9: I know how to do KT” and “Q11: I have experience conducting KT”. Wherever possible, items were kept that did not correlate heavily with other items. These items were also reviewed to ensure they captured the same or similar information.
For the remaining 59 items (version 1.0) we repeated the EFA, followed by an oblique rotation, producing as simple a structure as possible while permitting correlations among factors. This yielded 17 factors, with a KMO measure of 0.4 and a significant value for Bartlett’s test (p=0.000). The items, their medians, and inter-quartile range (IQR) for version 1.0 can be found in the supplemental material [Appendix 2]. Individual motivation, networks, prioritization, organizational climate, and resources were still captured by these factors. New constructs emerged in this model including institutional peer pressure, the process of conducting KT, and perceived value of KT.
All cross-loading items (i.e. items loading on more than one factor) or with a loading less than 0.5 were then dropped, resulting in the removal of 26 additional items (version 2.0). Table 3 shows each item included in version 2.0, the item’s median, and IQR. Some dropped items addressed individual motivation (e.g. Q13: I am passionate about conducting KT” and “KT activities have a positive impact on the health of communities”) and institutional climate (e.g. Q32: My institution provides opportunities for professional development in KT”; Q59: I have at least one mentor who conducts KT with the ministry of health”). We re-ran the factor analysis with 31 items, followed by oblique rotation, and identified seven factors. The model produced a KMO of 0.52 and a significant value for Barlett’s test (p=0.000). In this model one item (Q38: If I want to conduct a KT activity, I know where to find people in my institution who can help) loaded on two factors and was thus removed; in the resulting model, another item loaded to 2 factors and the item was dropped. A final set of 31 items was retained.
[Insert Table 3]
We re-ran the FA with the 31 retained items to identify a five-factor model. Items were selected based on the strength of the factor loading, uniqueness of the factors, the resulting scree-plot, and cross-loading criteria – and we selected 23 items for the five factors, with a N:p ratio of 5:1. Figure 1 displays the eigen value plot of the final factor model. For the final five factor model, the average communality (aka uniqueness) of selected items was 0.61. The items (p) per factor (r) ratio (p:r ratio) in the final model presented in Table 3 is 23:5 with N = 111. According to MacCallum et al., sample sizes of N = 100 and N = 200 are needed to estimate stable factor structure with 95% convergence for p:r ratio of 20:3 and 10:3 respectively if the average communality is low (less than 0.4) (31). If the communality is high (>0.4), as found in our study, a N = 60 is adequate for p:r ratio of 10:3 or 20:3 and will estimate the factor structure with over 99% convergence.
[Insert Figure 1]
The five final factors that emerged from the analysis were named, 1) Institutional Climate, 2) Organization Change Efficacy, 3) Prioritization and Cosmopolitanism, 4) Self-efficacy, and 5) Financial resources, based on their item characteristics and the underlying theories for those items (9). Factor 1 contains 6 items and was labeled ‘institutional climate’ because each item described aspects of their institution, colleagues, and leadership. Factor 2 contains 5 items and was labeled ‘organization change efficacy’ to capture organizational members’ shared beliefs in their joint abilities. Factor 3, ‘prioritization and cosmopolitanism’ which was also comprises of 5 items relates to internal and external institutional networks and priorities. Factor 4, comprises of 4 items, captures individual influencers of ‘self-efficacy’ including knowledge, skills, and time. Factor 5, ‘financial resources’, contains items related to internal and external budgets for KT activities.
Based on data collected during the stakeholder consultation process, these factors demonstrate face and content validity. That is, they appear to measure factors relevant to KT in these settings and represent the complex facets of the constructs
These five factors combined accounted for 69% of the total variance. The factor loadings, ranged from 0.40-0.77, are presented in Table 4; the intercorrelations between the factors ranged from 0.04 to 0.31. The final model observed a KMO measure of 0.554, and Bartlett’s test of sphericity was significant (χ2 (465) = 771.570, p < .000). Cronbach’s alpha coefficients were 0.78, 0.73, 0.62, 0.68, and 0.52 respectively. Factors 1 and 2 report an alpha above 0.7 which is traditionally acceptable (34, 35). Factors 3-5 report a lower alpha which can indicate a need for cautious interpretation. However, given the theoretical underpinnings for each of these factors related both to concepts of individual and institutional readiness, and the small number of items loading to each factor, we concluded the Cronbach’s alpha value are still helpful.
[Insert Table 4]