Demographics of MOOC registrants and survey respondents
Table 1 summarises and compares the demographics of the MOOC registrants and survey respondents revealing that the survey respondents are largely representative of the two cohorts of MOOC registrants.
Table 1. Comparison of MOOC participants and survey respondents demographic profiles
|
MOOC participants
|
Survey respondents
|
N
|
3,858
|
748
|
Gender
|
Female: 44%
Male: 57%
|
Female: 44.1%
Male: 55.9%
|
Age
|
Between 20-40 years: 77.5%
|
Between 26-40 years: 67.5%
|
WHO Region
|
African Region: 62.4%
South-East Asian Region: 17.7%
Americas Region: 9.9%
Eastern Mediterranean Region: 5.5%
European Region: 2.8%
Western Pacific Region: 2.2%
|
African Region: 69.4%
South-East Asian Region: 12.6%
Americas Region: 9.6%
Eastern Mediterranean Region: 1.6%
European Region: 4.1%
Western Pacific Region: 2.7%
|
Profession
|
Public health researchers: 45%
Public health officers: 15.5%
General practitioners: 11.1%
Students: 11%
|
Public health researchers: 31.2%
Public health officers 17.4%
Students: 15.3%
Teachers: 11.4%
General practitioners: 9.3%
|
Education level attained
|
Master’s degree: 41.5%
Bachelor’s degree: 24.7%
PhD: 10.5%
Medical Doctorate: 17.1%
|
Master’s degree: 57.1%
Bachelor’s degree: 25.6%
PhD/Doctorate: 17.3%
|
Certificate of completion obtained
|
Of the total initially enrolled: 30.15%
Of those who completed the course: 89.2%
|
70.6%
|
Demographics of MOOC registrants
Of the 3,858 registrants across the May and October MOOC offerings, 1,163 participants (30.2%) completed the assessments required to earn a certificate of completion. The MOOC registrants came from 115 different countries with the majority originating from the WHO African Region. The majority of registrants came from English speaking countries (87.2%) followed by Spanish (7.7%), French (4.6%) and Portuguese speaking countries (1.4%). Figure 1 below shows the regions of origin for the registrants.
Compared to men, women were more often public health officers (51.6% vs. 46.5%) and students (56.8% vs. 43.3%). Men were more often public health researchers (59% vs. 41%) and general practitioners (57% vs. 43%) than women.
Demographics of survey respondents
The demographic profiles have been combined for the May and October cohorts. The May cohort accounts for 45.4% of all responses and the October cohort for 54.6%. Of the 748 MOOC participants who took part in the survey, 525 (70.6%) completed the required assessments to earn a certificate of completion for the MOOC. 11% indicated that they participated in additional courses on IR following the MOOC. These additional courses included one-day workshops and symposiums, week-long training courses, online courses, one-year certificates, and Master’s degree programs. The survey respondents originated from 89 different countries and, as with the MOOC registrants, the majority of respondents originated from the WHO African Region (69.4%).
Knowledge of implementation research
Respondents were asked to indicate their reasons for taking the IR MOOC through a multiple choice question with an ‘Other’ option. The most commonly cited reasons included 1) To gain knowledge and understanding of IR (20.8%), 2) To apply the knowledge and tools in research (15.1%), 3) To apply the knowledge and tools in practice (13%), 4) For self-learning purposes (11.3%), 5) To further specialise in their field (8.5%), and 6) to obtain a certificate of completion (7.4%).
The majority of respondents considered their current knowledge about IR theory (frameworks, models and concepts) to be very good (15.6%), good (58.4%), or fair (23.1%). The IR MOOC contributed to their knowledge to a very large extent (18.1%), a large extent (50.5%), or to a moderate extent (26%).
In order to understand in further detail which areas respondents felt the IR MOOC had contributed to, they were asked what they considered as the most important insights or experiences gained from the course through an open-ended question. Of the 558 recorded responses to this question, six main themes were identified using Braun and Clarke’s six phase analysis. 28 These themes are as follows:
- Conceptual understanding of IR (N=185)
- Problem solving aspect of IR (N=71)
- Applicability or relevance of IR (N=68)
- Research process (N=58)
- Appreciation of stakeholder and community involvement (N=41)
- Program evaluation (N=36)
The following sections discuss each of these themes in further detail and provide representative statements.
Conceptual understanding of implementation research
The most common theme was that the IR MOOC contributed to respondents’ understanding of IR concepts, principles, theories, the importance of IR, and to their understanding why efficacious interventions fail in certain contexts. In relation to the importance of implementing workable solutions, one respondent wrote:
Solutions won't implement themselves. There is work to be done to make sure that the gap between know and do is bridged. (Public health researcher, Nigeria, Obtained certificate)
Another respondent revealed they had learnt the importance of bridging this know-do gap and highlighted that such gaps existed due implementation barriers.
I was able to find out why some programs failed in my State due to lack of data management and inappropriate penetration to the community by the implementers. (Public health officer, Nigeria, Obtained certificate)
Respondents indicated that the knowledge gained included an understanding of why interventions sometimes fail and the importance of including stakeholders in the implementation process to achieve the desired outcomes.
The fact that so many good interventions are always put in place to try to control/prevent NTDs [neglected tropical diseases] from occurring but yet not all were being effective against the particular diseases. This course helped me assess how best certain interventions would be followed to bring about the changes that were highly desired. It also taught me that many stakeholders should be involved in the work you do to achieve the best results. (Uganda, Researcher/microbiologist, Obtained certificate)
Such statements reveal that the MOOC was successful in educating participants about the core concepts of IR.
Problem solving aspect of implementation research
Respondents identified the importance of IR in identifying methods for solving community health implementation issues in particular contexts. The two representative statements below illustrate this point.
The ability to look at challenges from a broader perspective, visualising the different aspects of the problem and once identified, solutions are easily found. (Cameroon, Public health officer, Obtained certificate)
The most important insights or experiences that I gained from this course were implementation outcomes and implementation strategy. Now I can identify [an] implementation problem and propose a way to resolve it. (Public health researcher, Burkina Faso, Obtained certificate)
Methods for identifying “bottlenecks” were conveyed throughout the MOOC and respondents appear to have incorporated these learnings into their professional work.
Applicability or relevance of implementation research
Respondents’ comments indicated that they felt the MOOC IR highlighted how IR was applicable or relevant to their profession or research area. One stated:
Stakeholders’ involvement at all the stages of implementation research/activities. It was the most important insight because it was very impactful in my professional career. (Public health researcher, Uganda, Obtained certificate)
A second respondent outlined how the knowledge gained though the MOOC will positively impact on their work.
Having participated in community-based intervention programmes in my home country Nigeria, in the past, the knowledge I acquired in the course of the training revealed the causes of most of the challenges encountered and further gave me insight on how to carry out future intervention programmes in such a way that the community benefit more. (Student, China, Obtained certificate)
The applicability of the course material was also relevant to students and teachers as evident from the following statement.
The course helped me to restructure my PhD research work and my teaching materials for my students. (Student/teacher, Nigeria, Obtained certificate)
Respondents could therefore use the knowledge gained from the MOOC to make practical changes in their jobs.
Research process
In terms of research, respondents found the MOOC helpful in explaining in detail how IR is conducted. This includes designing, planning, and writing IR research proposals, applying for funding, writing publications, and developing research frameworks and methodologies. Representative statements include:
How to write a protocol; How to choose between quantitative/qualitative methods and why/how to use mixed-methods. (Public health researcher, Cameroon, Obtained certificate)
The most important insight gained is how to implement a plan accurately, communicate clear goals and anticipations, and evaluate programs fairly to learn what components are necessary to produce intended effect. (Human resource administrator/Researcher, Nigeria, Obtained certificate)
This finding reveals that respondents felt capable of developing research in IR following the MOOC.
Appreciation of stakeholder and community involvement
Respondents stated that the course had contributed to their understanding of the significance of community or stakeholder involvement, engagement or participation in the IR process. Statements include:
I learnt that involving community is vital for the successful implementation of interventions. (Public health researcher, Uganda, Did not obtain certificate)
Involve all the stakeholders from the beginning, make them partners, defining problems and providing solutions with them, improves the effectiveness of research and make it transparent, open and fun! (Research promotion and development administrator, Colombia, Obtained certificate)
As illustrated by the following statement, some respondents displayed a nuanced understanding of how strong community engagement helps maximize the chances that an intervention will succeed.
I understood that community mobilization will assist the community to identify the disease prevalent in the locality and available interventions targeted on diseases and the place they can assess care. This will lead to community involvement and participation, meaning that they will be involved in identifying their felt need. They will also select the trusted village health workers who will be trained to be involved in providing health services to the community. (Public health officer, Nigeria, Obtained certificate)
These findings indicate that the MOOC was successful in educating participants about the crucial role of stakeholder and community engagement in IR.
Program evaluation
A final theme relates to the significance of evaluating the effectiveness of implementation of intervention programs. Important learning cited by respondents are highlighted by the three respondents’ statements below.
How to use research tools to evaluate and improve on health programs. (Public health officer, Cameroon, Obtained certificate)
The evaluation of existing frameworks in order to introduce interventions that will improve on the status quo. (Student, Ghana, Obtained certificate)
How to assess the appropriateness of existing implementation strategies. (Student, Nigeria, Obtained certificate)
These responses indicate that those surveyed had engaged with the importance of evaluating an intervention’s effectiveness.
Changes in professional behaviour
In relation to changes in respondents’ work, the majority stated that the knowledge gained from the course had been valuable to a large extent (43.2%) followed by valuable to a moderate extent (27.8%) and to a very large extent (17.9%). The majority of respondents have had use for knowledge gained from the MOOC to a very large extent (12.6%), large extent (36.3%), or moderate extent (32%). The majority of respondents have had use for the knowledge gained from the MOOC monthly (30.6%), rarely (29.9%), daily (25.2%) or weekly (10.6%).
Respondents were asked whether they performed their role and responsibilities differently, as a result of participating in the MOOC. 69.1% agreed, while 30.9% disagreed. Of the 574 respondents, 330 elaborated on their response. The most common themes cited were:
- Research processes (N=94)
- Understanding and thinking about IR and issues related to research (N=73)
- Stakeholder and community involvement (N=46)
- Problem solving (N=31)
- Leadership (N=20)
- Involvement in IR projects (N=18)
- Teaching and training others (N=16)
- Program development, monitoring and evaluation (N=13)
Research processes
The most commonly cited difference in how respondents performed their role and responsibilities differently as a result of the course related to knowledge of and ability in research processes. This includes designing, planning, identifying research problems, developing research objectives, collecting data, presenting research outcomes, writing research proposals and publications, and applying for research grants. Three illustrative statements of this are presented below.
[The MOOC] helped me in developing a research proposal for my organization, where I used knowledge from this course. (Public health researcher, Nepal, Obtained certificate)
I produced more elaborate reports for projects, and better correspondence as I am able to identify relevant stakeholders. (Public health researcher, Ghana, Obtained certificate)
I have by myself completely written an IR proposal which has been funded. The course gave me the idea to include in my proposal participatory action-research and mixed-methods. I do think this played in my favor and made the proposal accepted by [name of organisation]. (Public health researcher, Cameroon, Obtained certificate)
Respondents’ self-reported changes in their research processes and outcomes reveals that the MOOC was successful in teaching IR research approaches.
Understanding and thinking about implementation research
The second most commonly found theme included respondents’ increased understanding and thinking about IR and both broad and narrow issues related to research. Representative statements include:
These courses have increased my knowledge in the implementation of research and allowed me to have a broad view of things. (Student, Congo, Did not obtain certificate)
From the knowledge gained from MOOC, I have changed how I perceived research implementation. (Internal medicine resident, Rwanda, Obtained certificate)
Such statements reveal the positive impact of the MOOC in shifting and developing participants’ thinking and understanding about IR.
Stakeholder and community involvement
Respondents commonly cited stakeholder and community involvement as an area in which they performed their role and responsibilities differently after participating in the IR MOOC. The following statements illustrate this point.
For all the interventions I plan, I promote the model of district-led-programming (DLP) to allow communities [to] lead the way’. (Public health officer, Uganda, Obtained certificate)
I always think of diverse stakeholders necessary for the success of a community-based intervention. (Student, Tanzania, Obtained certificate)
Comments reveal that respondents were aware of the importance of stakeholder and community context, and that including them increases the chances of the success of an intervention:
By following participants of my research to their local areas to learn challenges they face. (Public health researcher, Tanzania, Did not obtain certificate)
I often try to involve stakeholders in public health activities to promote acceptance. (Public health officer, Uganda, Obtained certificate)
The importance of stakeholder and community engagement was cited by respondents in relation to their understanding of IR as well as changes in their professional practice.
Problem solving
Respondents reported improvements to their ability to solve IR problems as a result of the course, as illustrated by the statements below.
The knowledge gained enables me to find simple yet innovative ways of solving problems. (General practitioner, Ghana, Obtained certificate)
I have implemented IR in my routine activities with problem solving in mind. (Public health officer, Nigeria, Obtained certificate)
Some respondents described how the knowledge gained from the MOOC had improved their abilities as a researcher and led to positive professional outcomes.
This is because the knowledge I acquired has transformed me into a more ardent researcher with ideas on how to identify and solve implementation problems. These ideas have contributed to designing an implementation research [study] identified by WHO/TDR experts, which gave me a chance of selection as one of the five finalists that attended [name of workshop]. (Public health researcher, Nigeria, Obtained certificate)
As problem-solving is a core component of IR, these statements by respondents reveal that the MOOC successfully communicated this point.
Leadership
A positive change cited by respondents concerned their increased abilities and roles in leadership. This includes leading research teams, managing research projects, taking on additional responsibilities, and contributing to policy making. Some respondents described the roles they were able to take on as a result of the MOOC.
Provide guidance and knowledge transfer to stakeholders and implementing partners on areas that need to be improved or strengthened in implementation of appropriate interventions, and documentation of the process. (Public health officer, Somalia, Obtained certificate)
One respondent provided an example of how this new ability contributed to their ability to lead research teams nationwide.
As a senior research advisor I am using the skills I have acquired to develop concept notes and protocols for IR and guide research teams across the country. (Public health researcher, Ethiopia, Obtained certificate)
For others, the leadership abilities assisted them to improve the implementation of public health programming.
As a programme manager, it has helped me guide programme implementation in identifying gaps and steps/required interventions to achieve desired implementation outcomes. (Public health officer, Nigeria, Obtained certificate)
Involvement in implementation research projects
Respondents identified increased involvement in a range of IR projects as a result of the MOOC. This is illustrated by the following statements.
I have conducted TB implementation research in my country. (Research and development/Private sector, Sudan, Obtained certificate)
I have been part of various teams in Nigeria developing and working on implementation research. (Public health officer, Nigeria Did not obtain certificate)
Some respondents described how the MOOC had refreshed their knowledge and enabled them to apply it to IR projects.
My MPH training included some coursework on implementation science (IS) but the MOOC really helped me refresh my knowledge and apply IS lens to my role at [organisation name]. My role is focused on managing global health security project implementation across several countries in Africa and Asia. (Public health officer, US, Obtained certificate)
Teaching and training others
Respondents indicated that the course had assisted them in their roles and responsibilities regarding teaching and training.
As a programme manager, it has helped me guide programme implementation in identifying gaps and steps/required interventions to achieve desired implementation outcomes. (Public health officer, Nigeria, Obtained certificate)
Some participants stated that the course had improved their thinking around teaching and training.
I now design and teach research in a manner that its findings could be implemented. (Teacher, Nigeria, Obtained certificate)
Greater clarity when advising students how to execute their investigations in the field of tuberculosis in the Master of Epidemiology at [university name]. (translated from Spanish) (Teacher, Colombia, Obtained certificate)
Others indicated that the MOOC had increased their confidence in this area.
I feel more confident to train others in IR. (Public health researcher, Colombia, Obtained certificate)
Program development, monitoring and evaluation
The MOOC increased respondents’ skills within general program development, monitoring and evaluation.
I ensure that the interventions targeted at the diseases of poverty are utilized by the communities. (Public health officer, Nigeria, Obtained certificate)
I have been able to assess bottlenecks from a systems point of view and realize how these bottlenecks can or could be solved with implementation research which helps us to understand why they are occurring in the first place. (Student, Malawi, Obtained certificate)
Some respondents indicated they were able to transfer this knowledge to other areas.
After taking the course, I used knowledge gained in assessing the Adolescent Sexual and Reproductive Health program running in my district and I used findings to change some methods in adolescents education and care in school and at health facilities in my district. (Public health officer, Ghana, Obtained certificate)
A second participant stated:
In that I turn every public health program (food safety programs for instance) around to detect any possible gap in its implementation and find solutions to them before getting it out to its consumers or beneficiaries. (Regulatory officer, Ghana, Obtained certificate)
Such findings are important in that they confirm the usefulness of the IR MOOC in developing participants’ ability to develop, monitor and evaluate programs that are not focused on infectious diseases of poverty.