The Quality Of Cluster Randomized Controlled Trials Of Pediatrics Should Be Improved: An Integrative Literature Review

Background: The number of cluster Randomized Controlled Trials (cRCTs) have been increasing and application in pediatrics is also growing. However, we did not know the quality of cRCTs in pediatrics and no previous study has comprehensively analyzed cRCTs in pediatrics. To investigate the status of cRCTs in pediatrics and analyzed inuencing factors of the methodological quality. Methods: Through a search of 10 highest-impact-factor journals from PEDIATRICS and MEDICINE, GENERAL & INTERNAL, we identied cRCTs published from 1 January 2015 to 7 June 2020. The associations between study factors and grade of ROB (Risk of Bias) quality were analyzed by Pearson’s chi-squared test. Results: A total of 73 cRCTs were included. For the number of cRCTs, it has uctuated in the past ve years. For the International Classication of Diseases Eleventh Revision (ICD-11), 34.2% cRCTs could not be classied, of which more focused on vaccines; 15.1% cRCTs were classied into "X Extension Codes", of which more focused on vaccines and school bullying. For research settings, studies were concentrated in schools, communities, hospitals, clinics, and kindergartens, of which schools (36.5%) were the most. The number of clusters ranged from 1 to 1533 (M=28) and the participants ranged from 135 to 190,238 (M=1903). For the results of ROB assessment, 31 cRCTs were in high-quality, 28 were in low-quality, and 14 were unclear. The majority of cRCTs provided ow diagrams (87.8%), registered (98.6%), and made a distinction between primary and secondary outcomes (86.5%). Conclusions: The further improvement of the quality of cRCTs in pediatrics is needed, particularly in the terms of randomization, allocation concealment, and blinding. Writing and publishing the protocol can enhance transparency. Comply with of CONSORT statement for future studies is needed.


Background
Because of the strong argument strength, randomized trials usually help researchers determine the e cacy of interventions in evidence-based medicine [1] . The multi-center design is a common method used in clinical trials, particularly in randomized controlled trials (RCTs). It is also widely used in evaluating the intervention's e cacy because of interventions in clusters [2] , which makes it possible not only to avoid contamination [3] , but also to combine the advantages and characteristics of RCTs. Based on the above, and take into account the particular characteristics of children, cRCTs are also particularly appropriate for applications in pediatric studies. In recently, the number of cRCTs have been increasing and application in pediatrics is also growing. A crosssectional survey in 2021, however, nd that both the methodological quality and reporting quality of cRCTs need to be improved, particularly the selection bias of methodological quality [4] . Currently, no previous study has comprehensively analyzed cRCTs in pediatrics; indeed, no study has identi ed what characteristics affecting the quality of cRCTs. Therefore, in this survey, cRCTs in the high-quality journals in the eld of PEDIATRICS and MEDICINE, GENERAL & INTERNAL were included to summarize the characteristics, to assess of quality based on the Cochrane risk of bias assessment tool (ROB), and to analyze the in uencing factors of the methodological quality of cRCTs in pediatrics. terms were "pediatric*", "paediatric*", "child*", "infant*", "adolescent*", "baby", "babies", "trottie*", "youth*", "kids", "toddler*", "preschool*", "preschool*", "kindergarten*", "kinder-garten*", "girl*", "boy*", "student*", "junior*", "juvenile*", "neonat*", "newborn*", "teenager*", "pubescent", "preterm", "puberty", "young*"; "cluster", "crossover trial", "cluster-crossover trial", "cluster crossover trial", "cluster-randomized", "cluster randomized", "cluster-randomised", "cluster randomised", "cluster RCT".

Selection Criteria
Studies ful lling all the following criteria were considered eligible: (1) the study was a cRCT; (2) the study subject was about newborn, infant, toddler, child, and adolescent; and (3) studies published in Chinese or English.
The following types of studies were excluded: (1) conference proceedings, abstracts, letters, or protocols; (2) studies for which the full text was not available; (3) studies were duplicated; and (4) RCTs at the individual level.

Study Selection
After eliminating duplicates, four researchers independently in pairs in two steps as described below. Before the formal selection, a training exercise of a random sample of 200 citations was conducted to ensure the reliability and feasibility of selection, until adequate agreement on the selecting methods was reached. Firstly, the titles and abstracts of the records were independently screened by four researchers based on selection criteria, and then the full texts of potential suitable articles were retrieved for further assessment. Disagreements were resolved by discussion or consultation with a third researcher.

Data Extraction
Using a pre-developed data extraction form, six researchers independently in pairs extracted study characteristics including: (1) Basic information: publication year, title, journal of publication, country of the rst author, funding, con icts of interest, study topic, ICD-11 classi cation, the country in which the study was conducted, study setting, and whether multi-center; (2) Study design: whether the trial registered, wrote a protocol, published protocol, had a diagram, conducted repeat observations, conducted evaluator training, conducted additional analyses, distinguished primary and secondary outcome indicators (yes or no), and type of intervention, number of centers, number of participants.

Quality of Evidence Assessment
Six researchers independently evaluated the methodological quality of included studies using the Cochrane risk of bias assessment tool [5] in pairs. Disagreements are resolved by consensus or consultation with a third researcher. The overall Cochrane ROB tool grades methodological quality of included studies as high, low, and unclear [6,7] according to the risk of bias of following 8 items: sequence generation (randomization) , allocation concealment, blinding of participant, blinding of personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting and other sources of bias. See Table 1 The grade of ROB for details. Table 1 The grade of ROB The grade of methodological quality

High Quality
Two or more of the ve key items a have low bias, and three common items b without high bias.

Low Quality
One or less of the ve key items with low bias, and regardless of other common items.

Unclear
Other conditions. a: The ve key items included sequence generation, allocation concealment, blinding of participant , blinding of personnel , blinding of outcome assessors.
b: three common items were incomplete outcome data, selective outcome reporting, and other sources of bias.

Statistical Analysis
We established the database with Microsoft Excel 2019 software, and descriptive statistics were performed by SPSS software, version 25.0 (Statistical Package for the Social Sciences version 25.0). Qualitative variables were described by absolute numbers (percentages). Quantitative variables were expressed as mean ± standard deviation (SD). We then conducted chi-square test to assess the associations between study factors and rates of ROB. A difference was considered statistically signi cant effect when the P-value equal to or less than 0.05.

Financial Support
The funding source had no role in design of study, the collection, analysis, interpretation, and the decision to approve publication of the nished manuscript.

Ethical Approval
Not applicable. All the work was developed using published data.

Study Selection
We identi ed 5,147 records from database. A total of 73 cRCTs were included eventually. More details of the search and selection are shown in Figure 1 Flow chart summarizing the process of identifying the eligible studies..

Study Characteristics
The number of included cRCTs uctuated from 2015 to 2020.  (11,15.1%) and growth and obesity of child and adolescent (11,15.1%). According to ICD-11, most cRCTs cannot be classi ed (25,34.2%), of which more focused on vaccines (5, 20%); many cRCTs focused on the most was "X Extension Codes" (11,15.1%), of which more focus on vaccines (6, 54.5%); then a part of studies concerned about "Endocrine, nutritional or metabolic diseases" (9, 12.3%) and "Certain infectious or parasitic diseases" (7, 9.6% Table 2 Characteristics of included studies at the end of the text.  Table 3 The results of ROB assessment.

Comparisons of ROB Between Groups
We evaluated the association between each of the trial characteristics on trial quality. The Chi-square test showed that produced a protocol has a statistically signi cant impact on the methodological quality of cRCTs (OR=0.261, P=0.014), and other trial factors had no statistically signi cance. See Table 4 Chi-square analysis results of each research factor at the end of the text for details.

Discussion
In this cross-sectional survey of cRCTs published in 10 top journals in recent years. We found that the number of included cRCTs uctuated in past ve years, the elds of concern were wide, and the methodological quality of cRCTs still needed to be improved.
We found that the number of cRCT literatures in pediatrics in recent ve years uctuated, which may be related to the reason that we only included literatures published in Chinese and English, and exclude the literatures that inability to obtain full-text and those none periodical. The reasons listed above may affected the amount of literature in different years, thus affecting the overall trend. Another reason is that we limited retrieval time to June 2020. Whatever the publishing trend, we can nd that the pediatric eld has continued to pay attention to the application of cRCTs in recent years. According to the data released by WHO, by the end of 2018, the top three diseases in the mortality of children under the age of ve were acquired immunode ciency syndrome, diarrhoeal disease, and tetanus [8] . It can be seen that children's health and life safety still threatened by infectious diseases. However, vaccination is the key to reduce child mortality in low and middle-income countries [9][10][11] . In this study, we found that most cRCTs pay high attention to vaccines, which is consistent with the above situation. In addition, obesity has always been the focus of public health all the world [12,13] , which is also consistent with the more attention paid to the development and obesity of children and adolescents in this study.
In this study, we found that the quality of cRCTs still needs to be improved. There were few descriptions of randomization, allocation concealment, and blinding in cRCTs. The risk of bias is generally high, from previous studies we found that more than half of RCTs do not implement appropriate blinding or undescribed blinding [14][15][16][17][18] . A study in 2021 showed that a better methodological design is needed to improve the quality of cRCTs, such as calculation of sample size and balance randomization [19] . Therefore, we can conclude that the methodological quality of cRCTs still needs to be improved. In addition, the reporting quality of cRCTs should be improved too. From the previous study, we found that the reporting of RCTs is still insu cient [20] and imperfect [21] , which is di cult to provide readers with necessary information. Published RCTs rarely reported the details of blinding and allocation concealment [22][23][24] . The same as cRCTs, a study published in 2021 found that the current reporting quality of multicenter RCTs is not optimal when evaluating the reporting quality of cRCTs from 1975 to 2021 [25] .
With regard to the in uencing factors, more than half of the literatures in this study wrote a protocol. The protocol is the basis for clinical trial planning, implementation, reporting, and evaluation [26] . Furthermore, the Consolidated Standards Of Reporting Trials statement 2010 (CONSORT) extension to cluster randomized trials stated that the protocol should be written in advance, registered on the clinical trial registration, and the complete version of the protocol should be provided in the text. Moreover, strictly following the CONSORT statement can improve the reporting quality [27] and provide readers with more comprehensive information. In this study, we found that the quality of trials with protocol was higher than that of without, which was consistent with the above researches. cRCTs are prone to loss of follow-up and missing data during the implementation of the study due to a large number of participants and long follow-up time. In this situation, some studies suggested to reduce bias and improve the quality of studies by using intention analysis and sensitivity analysis in the analysis of cRCTs [28] . This study found that most of the pediatric cRCTs reported the owcharts and adopted the additional analyses. However, the quality difference between studies was small, because included studies were from high impact factor journals, thus we didn't nd out the impact of the owcharts or use of additional analyses on the research quality.
Based on the ndings of this study, we put forward the following suggestions for carrying out pediatric cRCT researches, which is also applicable to other disciplines to a certain extent. Firstly, we suggest that cRCTs should write a protocol to help the effective implementation of the research and increase the transparency of the research. Secondly, we suggest researchers should fully master the methodology of the trial, especially the implementation of a strict randomization, allocation concealment and blinding before the implementation of cRCTs, epidemiological professionals or methodologists can also be included in the trials for full guidance. Finally, we suggested to increase the training and use of CONSORT and its extended version for researchers, and further urge the standardized reporting of the trials.

Strengths and Limitations
This study has several strengths. We performed a comprehensive analyses of cRCTs in the eld of pediatrics for the rst time as we know. The blinding is divided into the blinding of participant, blinding of personnel and the blinding of outcome assessors, instead of the joint evaluation method when assessing the risk of bias of the study. In this way, we increased the reliability of the evaluation results [29] . In order to provide references and suggestions for future research, we also analyzed the factors affecting the ROB of cRCTs. At the same time, this study has several limitations. Firstly, the evaluation of the current research hotspots in the eld of pediatrics may be affected, that many included studies could not be distinguished to the speci c disease classi cation according to ICD-11, of which 34.2% cRCTs involved diseases that could not be classi ed, and 15.1% cRCTs involved diseases classi ed as "X Extension Codes". In addition, we may omit some important factors that affect the research quality, due to the cRCTs included in this study are all from high impact factor journals and the methodological quality heterogeneity between studies is small. Furthermore, the analysis results may be affected by publication bias, for this study excluded the literatures published in no Chinese and English.

Conclusion
Overall, nearly half of the pediatric cRCTs published in journals with high impact factors in the past ve years are of high quality. However, the ve key items of the ROB still need to be improved. As to increase the standardization and normalization of cRCTs and eventually improve the quality, we recommended to write a protocol before the experiment, strengthen methodological training for researchers, and in accordance with the CONSORT statement when reporting. Flow chart summarizing the process of identifying the eligible studies.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.