Massive Open Online Courses (MOOCs) Dropout Rate in the World: A Systematic Review Protocol

Recently, massive open online courses (MOOCs) have received increasing popularity throughout the world. Regardless of the subject taught and the university providing the course, the dropout rate of MOOCs is one of the most important challenges ahead. This systematic review will search MEDLINE/PubMed, Scopus, Web of Science (Clarivate Analytics), Embase (Embase.com), ASSIA, CINAHL, Education Research, BEI, and Eric databases systematically according to predened criteria without language restrictions to retrieve prospective and retrospective observational studies conducted between the 1 st of January 2000 and 30 th of March 2020 which evaluated the frequency of leaving MOOCs throughout the world. In the absence of severe methodological heterogeneity, the data will be combined and a meta-analysis will be performed.


Background
Massive open online courses (MOOCs) have enjoyed increasing popularity in the world for the last 10 years (1)(2)(3)(4)(5)(6), and with the growth of the Internet and educational technologies, the number of people joining MOOCs is increasing every day (1,6,7). According to the European open training report, which provides MOOC-related statistics in European countries, there was a 130% increase in the number of MOOCs from September 2014 to September 2015 (8).
It appears that the need for MOOCs is apparent to universities and governments, and currently, these projects are encouraged and receive funding (1).
By de nition, a MOOC is a course that is open, meaning there are no barriers to entry, and entails no special educational costs or features. It is provided online through the Internet, and many people can connect to its educational environment (4).
In practice, dropout of MOOCs, regardless of the subject taught and the university providing the course, is among the challenges MOOCs face (1-10). Online course dropout hinders universities from meeting the minimum expectations of such education (7). Course dropout has also been a concern to educational institutions because it is a waste of social, academic, and economic resources when a student begins a course but does not nish it (11).
Evaluating the global status of course dropout can provide crucial information that will help save resources and design new generations of MOOCs (9). The present systematic review addresses the gap in the literature on the MOOC dropout rate in the world.
According to our knowledge, in a study in 2017, titled "How do we model learning at scale? A systematic review of research on MOOCs", Josksmovic et al. conducted a systematic review of approaches to model learning in MOOCs and reviewed studies on the prediction and measurement of students' staying in MOOCs. A total of seven databases were examined by computer search covering from 2012 to 2015 and nally, 38 studies were analyzed. As one of the objectives of the analysis, the number of enrollments and completions of courses was also addressed. Of these 38 studies, 12 were extracted from journals, and others were presentations at conferences. The number of courses examined in the studies varied from 1 to 133. The highest and lowest enrollment rates in a course were 513098 and 1189 people, respectively. Among the 13 studies that mentioned the exact number of enrollments and completions of courses, the highest and lowest dropout rates were 98% and 53.3%, respectively. This study included no meta-analysis (9).
Furthermore, three review studies have reported related ndings as follows: In a study titled "Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017", Paton searched 10 databases and the results showed that the dropout rate was 65% among Vocational Education and Training (VET) learners, 71% in Australian universities, and 72% in international universities. This study mentioned poor transferability and the narrow timeframe as its limitations (12).
Another study, titled "A review of literature on the reasons that cause the high dropout rates in the MOOCs", by Bezerra did not mainly aim at estimating course dropout rates, but mentioned a preliminary study that examined 91 courses in 2014 and found a dropout rate of 90% (8).
The third review study, titled "MOOCs completion rates and possible methods to improve retention: A literature review", in 2014 examined the causes of MOOC dropout and solutions for reducing it. The authors referred to a study by Jordan (2013), which examined 50 MOOCs with 50000 enrolments and reported a course dropout rate of more than 90%; they also referred to a study by Meyer (2012), which reported an 80%-90% dropout rate, with only 2% of enrollments culminating in receiving a certi cate. The authors also referred to other studies in the eld and reported a course dropout rate of 88.5% for a machine-learning course by Coursera, 94% for a bioelectricity course by Duke University, and 98.5% for a pattern-oriented software architecture course (13).
The present study aims to combine data from studies conducted since 2000 (MOOCs were introduced in 2008), identi ed through searching various databases and gray literature with no language restrictions, to estimate the global rate of MOOC dropout systematically after qualitative evaluation of the studies.

Objectives Primary objective
The primary objective of this systematic review is to estimate the global frequency of MOOC dropout Secondary objectives 1. Estimation of the global frequency of MOOC dropout by age group, gender, and job type.

Methods/design
This protocol is prepared according to the Centre for Reviews and Dissemination (CRD) guidelines and will be reported according to the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines. The selection process of the studies will be reported according to the Preferred Reporting Items for Systematic review and Meta-Analysis-Protocols (PRISMA-P) 2015.
Eligibility Criteria of Primary Studies:

Inclusion and exclusion criteria
Type of the study: This systematic review will include prospective and retrospective observational studies that evaluate the frequency of leaving MOOCs throughout the world. Studies should contain the frequency (%) of the people who left courses. No restriction on sample size and language will be applied for primary studies.
Type of participants: All students who have participated in a MOOC, with ages equal or above 18 years (adult), in either gender, in all geographical regions, and with all job types, will be included in this study. Other methods used to nd relevant studies are manual search of gray literature (thesis, conference papers, and organizational reports) and contacting experts to obtain information on their relevant unpublished studies and related conferences.

Search of relevant Internet resources
To access more data, we will conduct a search of Class Central (http://classcentral.com).

Time interval of search
All studies conducted between the 1 st of January 2000 and 30 th of March 2020, will be included.

Screening and Selection
Upon completion of the search process, two of the authors (ARS and MM) will review the titles and abstracts of the studies according to the inclusion and exclusion requirements at the screening level, and will identify and pick the appropriate studies. Studies with su cient data in one or more aspects of the inclusion criteria will be included and, in the subsequent stage, a nal decision will be made according to their full text.
Eventually, in the next step, two of the authors (ARS and AAK) will independently review the full text of the selected studies to determine the nal included studies.
Discordances in any step of the process will be resolved by consensus, and if the disagreement is not resolved, the opinion of a third expert will be used to resolve the case.

Assessment of the risk of bias in included studies
Two of the authors (ARS & MM) will independently use the Newcastle-Ottawa Scale (NOS) for observational studies (for cross-sectional, retrospective, and cohort studies) to assess the quality of nal included studies. This scale has eight items covering aspects of selection, comparability, and outcome of the studies.

Data extraction
For the nal included studies, two of the authors (ARS and MM) will independently extract the following summary data: rst author's name, study design, country, sample size, demographic variables of the participants, year of publication, type of the university, participants' jobs, level of the online course, duration of the MOOCs, and MOOC dropout rate (%) and its causes.

Statistical analysis
Pooled analysis The pooled frequency (%) of MOOC dropout will be calculated in the process of meta-analysis. The combination method will be based on methodological similarities of the included studies by the xed effects model or random effects model. Forest plots will be made for all studies to show the separate and pooled frequencies and their corresponding 95% CIs. Stata V.13.1 (Stata Corp) software will be used in the present study. If meta-analysis is not possible due to severe methodological heterogeneity, a narrative qualitative discussion, based on ndings from individual studies, will be presented.

Assessment of heterogeneity
The Q-statistic test and I 2 statistic and their corresponding 95% CIs will be used to assess the statistical heterogeneity of the frequency values in the included studies. The degree of heterogeneity will be determined according to the Cochrane Handbook (26). The level of statistical signi cance for the Q-test will be set at p< 0.05.

Subgroup analysis
Subgroup analysis or meta-regression, if su cient data are available, will be used as appropriate to investigate the effect of statistical heterogeneity. In this study, variables such as age, sex, and duration, types, and degree levels of courses will be used in subgroup analysis.

Sensitivity analysis
The leave-one-out method will be used for sensitivity analysis. If one of the combinations (K-1) of the studies shows a result different from those of the others, we will carefully consider the features of the studies involved.

Assessment of publication bias
If there are at least ten studies in the nal selection, the funnel plot and Begg's and Egger's statistical tests will be used to evaluate publication bias.

Discussion
Dropout rate of MOOCs is one of the most challenges that universities may encounter. On the other hand the number of people joining MOOCs is increasing. Also it is apparent that these projects need investment both in human resources and economic resources. This systematic review will help universities which are deciding on launching these kinds of platforms to extend their view and correct their expectations about staying in the course. This study also will provide information which will help universities save their resources or maybe design their MOOCs differently. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.