The Potential Inuence of Absence of Blinding Method: Secondary Analysis From Randomized Controlled Trials Concerned Acupuncture for Depression

To evaluate randomized controlled due Trials assessing acupuncture for depression published by March 2020 were searched from China National Knowledge Infrastructure database. Through matching, we reassembled the trials of comparing experimental acupuncture and control acupuncture. Meta-analysis with post-treatment HAMD (Hamilton Depression Scale) of the newly reassembled trials was conducted. Changes of HAMD of all included groups were also analyzed, the between groups difference was then analyzed by t-test. All statistical analyses were performed using RevMan 5.3 software. Thirteen pairs of studies were matched from 63 included trials. Re-analysis from the paired studies showed obvious difference of HAMD scales between the experimental and control acupuncture (Mean Difference was -2.95 for HAMD-17, -5.55 for HAMD-24). Re-analysis from all the included groups also showed the statistical difference between groups at four weeks of HAMD-17 (P=0.01), and four/six weeks of HAMD-24 (P<0.01).


Abstract Background
To evaluate the possibility and impact of performance and detection bias in randomized controlled trials due to lack of the blinding method.

Methods
Trials assessing acupuncture for depression published by March 2020 were searched from China National Knowledge Infrastructure database. Through matching, we reassembled the trials of comparing experimental acupuncture and control acupuncture. Meta-analysis with post-treatment HAMD (Hamilton Depression Scale) of the newly reassembled trials was conducted. Changes of HAMD of all included groups were also analyzed, the between groups difference was then analyzed by t-test. All statistical analyses were performed using RevMan 5.3 software.

Results
Thirteen pairs of studies were matched from 63 included trials. Re-analysis from the paired studies showed obvious difference of HAMD scales between the experimental and control acupuncture (Mean Difference was -2.95 for HAMD-17, -5.55 for HAMD-24). Re-analysis from all the included groups also showed the statistical difference between groups at four weeks of HAMD-17 (P=0.01), and four/six weeks of HAMD-24 (P<0.01).

Conclusion
Effectiveness of acupuncture for decreasing the severity of depression was superior in trials that considered acupuncture as experimental treatment, showing performance and detection bias caused by the absence of blinding method is of great signi cance in acupuncture trials.

Background
The uncertainty principle attaches great signi cance to the uncertain state of therapeutic effects of the methods used in clinical trials, which is essential for controlling bias. A case in point is a study by Benjamin Djulbegovic discovered that clinical trials sponsored by the pharmaceutical industry result in biased ndings, either due to selective report of studies with non-equivalent arms or publication of lowquality papers, wherein unfavorable results are incompletely described [1].
However, it is di cult for researchers to eliminate the participants' preference of intervention effect. Therefore, in order to avoid the in uence of subjective will or preference on the research results, the study design principle came into being. Among these principles, blinding method is widely used in randomized controlled trials to avoid performance bias and detection bias [2]. The premise of success in implementing blinding method is that the experimental intervention has a reasonable placebo control, which is almost impossible for most non-pharmaceutical therapies, such as acupuncture. Thus, in case of absence of blinding, the researcher's values or preference may affect the procedure of the study and the results [3]. Study by Lingling Yang [4] categorizes several clinical trials according to therapy and grouping, and whether acupuncture has signi cant effect varied remarkably.
For centuries, literatures published about traditional Chinese medicine have been all about personal experiences. Although the rise of clinical epidemiology and evidence-based medicine has greatly improved researchers' understanding of the rigor of research design, trials' methodological quality has not improved substantially [5]. A case in point is a meta-analysis study by Mike Armour suggested that [6] 'A test for between acupuncture and control group differences showed that trials from China had signi cantly greater reductions in the severity of depression compared to those undertaken outside of China (Mean Difference = 0.58, 95%CI 0.25 to 0.91, P < 0.001). This may be caused by selection and reports of favored studies or other bias originated from non-equivalent control.
Performance bias and detection bias, originated from researchers' preference of a certain therapy and insu cient use of the blinding method. To evaluate whether and how these two biases may affect the results of acupuncture clinical trials, we re-analyzed the data from the eligible acupuncture trials through meta-analysis method.

Eligibility criteria
Published randomized controlled trials with completely reported continuous data were concerned to be included. "Completely reported" means that mean of the outcome measurements for each group must be reported with their standard deviation (SD) or 95% con dence interval (CI) The target diseases are primary or secondary depression diagnosed in accordance with recognized criteria (e.g. Diagnostic and Statistical

Manual of Mental Disorders Fifth Edition).
Acupuncture is the intervention of this study. By comparing the groups that were designed as experimental treatment and as control, we were able to explore whether the researchers' expectations will affect the results of acupuncture therapy for depression. Trials which employed acupuncture as experimental treatment may choose no treatment or standard treatment as control, thus we ltered out the trials which compared acupuncture to no treatment or western drugs (such as uoxetine hydrochloride capsules, etc.).
In this case, acupuncture was considered as "experimental treatment" and this arm of the trial was counted as intervention/experiment group in our review. Trials which used acupuncture as control treatment were mostly aimed to investigate the effect of another non-pharmaceutical therapy (e.g. Cognitive therapy, psychotherapy, music therapy. Etc.) for depression, or to verify the effect of a special acupuncture (special manipulation or needle type) for the disease. In this case, the routine acupuncture group was identi ed as control arm in our review.
Primary outcome of this study was the score of Hamilton Depression Scale (HAMD). Literatures with inaccessible data, as well as the piratical documents would be excluded.

Searching strategy
The Chinese National Knowledge Infrastructure Databases was searched, including the unpublished dissertations. The searching items included 'Acupuncture' AND 'Depression' in the scope of Title/Abstract, and "Random" in full text.

Study screening and data exaction
Two reviewers (RTW and LJC) independently screened the literature to identify trials that potentially meet the above inclusion criteria. Any disagreement was resolved through discussion with a third reviewer (HJC). A predesigned form was used to extract data from the included trials. Extracted information included: i) General information, including document number, title, rst author, year(s) conducted, location (city, country), source, etc. ii) Participants information, including diagnostic criteria, inclusion criteria, exclusion criteria, source, sample size, age, gender, disease course. iii) Treatment information, such as the treatment principle, types of acupuncture, information of the acupuncturist, treatment duration, points selection, treatment frequency, co-intervention, etc. iv) Outcome measures, including the type of HAMD scales (17-scores or 24-scores), changes in HAMD score, post-treatment HAMD scores, time of the measurement.

Data analysis
All statistical analyses were performed using RevMan 5.3 (The Cochrane Collaboration) software.
Since data of "experimental acupuncture" and "control acupuncture" to be compared in our study were get from different trials, we tried to make the two acupuncture groups comparable by matching them. After completing the data exaction, we selected the matching experimental and control acupuncture groups to reassemble trials. The rst principle of matching is that the two groups of patients have the same type of depression, type of HAMD scale, and the treatment frequency. Besides, there should be no statistical difference (P > 0.05) between two groups in baseline information, including age, sex ratio, and the pretreatment HAMD scores. The prescriptions of the acupoints were also matched, however, we could only guarantee the equality of the main treatment principle between the reassembled groups since most of the trials were with individualized points selection. We then re-analyzed the difference of post-treatment HAMD scores in the new paired-study through RevMan 5.3. A meta-analysis with mean difference (MD) and 95% CI was used to present the results. I2 value re ect the statistical heterogeneity among trials. Data from the included groups which failed to be matched were not included in this analysis.
Furthermore, we also analyzed data from all the included groups regarding to the changes of HAMD score pre and post treatment within groups. Then, we separately pooled the data from experimental acupuncture group and control acupuncture group with subgroup meta-analysis. Subgroup was classi ed according to the type of HAMD scale and treatment duration. Difference of the pooling result between groups was then analyzed by t-test.

Study selection
Totally 1659 clinical trials were retrieved through preliminary searching. After reading the title and abstract, 869 trials were screened out. And further another 700 trials were excluded after reading the full text. Sixty-three randomized controlled trials  were nally assessed to be eligible (Fig. 1).

Re-analysis from the unpaired-study
The results of pooling data from both experimental and control acupuncture groups from 61 studies showed there was signi cant statistical difference of changed HAMD-17/24 scores after four weeks and six weeks treatment [7-31, 33-55, 57-69]. Details of the characteristics of the unpaired-studies were listed in Table 2. The t test results (see Table 3

Discussion
According to our study, the effectiveness of acupuncture for decreasing the severity of depression was superior in trials which considered acupuncture as experimental treatment compared to those of the acupuncture as control. The difference of post-treatment HAMD-17 scales and HAMD-24 scales were average − 2.95 cores and − 5.55 cores, which were both statistical and clinical meaningful for relieving depression.
This study attempts to quantify the impact of performance and detection bias in acupuncture clinical trials. The application of paired meta-analysis can reduce the system error of indirect comparison as much as possible, and re-analysis from the unpaired-study further con rmed the stability of the results.
The data of this study can explain to some extent the consequences of destroying the uncertainty principle caused by the lack of blinding method in the non-pharmaceutical therapy researches represented by acupuncture.
However, since we used indirect comparisons, the strength of the evidence was weakened. Although we used pairing method to reduce the in uence of confounding factors, the results of meta-analysis had a certain degree of statistical heterogeneity due to the inconsistency of acupoint selection between acupuncture studies.
As the result showed in the study, we suggest that consideration should be given to existing evidence of acupuncture for depression. When acupuncture is used in the experimental group, the effect of acupuncture may be overestimated; but it may be under estimated when acupuncture appearing in the control group. It is suggested that the actual effectiveness of acupuncture should be carefully considered when referring to the corresponding evidence in clinical practice. Similar consequences may appear in other diseases, as doctors' personal preference could lead to performance and detection bias, therefore implicates study results. Moreover, it can be speculated that detection bias could cause due to patients' self-assessment, which is highly subjective.
According to this study, blinding method, applied with extra means to ensure its success, is the key solution to avoid performance and detection bias during acupuncture clinical trials. We suggest that future trials should either use adequate random allocation and concealment, blinding the outcome assessors and statisticians, or improve the monitoring mechanism, so that the research designer does not intervene in the implementation process of intervention and does not directly contact patients, so as to reduce the impact of bias on the research. However, blinding method is hard to be used in the nonplacebo-controlled trial. Thus, some special types of randomized trials can be used in acupuncture clinical trial. For example, sequential multiple assignment randomized trial can improve external authenticity and get closer to the real treatment process through building and comparing dynamic treatment regimens [70][71][72]. In previous studies, we also proposed the feasibility and advantages of partially randomized patient preference trial adopted in evaluation of non-drug therapy (e.g. Acupuncture) by analyzing the application status of modi ed clinical trials considered patients' preferences [73,74].

Conclusions
This study found that the effectiveness of acupuncture for decreasing the severity of depression was superior in trials those employed acupuncture as experimental treatment, which showed that performance and detection bias caused by the absence of blinding method played an important role in acupuncture clinical trials. Availability of data and materials

Abbreviations
The authors con rm that the data supporting the ndings of this study are available within the article and its supplementary information les.

Competing interests
The authors declare that they have no competing interests.

Funding
Hui-Juan Cao is supported by the National Natural Science Foundation of China (81804000). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Authors' contributions RTW and LJC analyzed basic characteristics of trials and analyzed data, and were major contributors in writing the manuscript. JPL and YTF assisted with the research methods and result explanation. MH participated in research designing, data analyzing and interpretation of indicators. HLC ensure that original data, original gures, materials and code upon which the submission is based are preserved following best practices in the eld so that they are retrievable for reanalysis; and con rm that data, gures, materials and code presentation accurately re ects the original; and foresee and minimize obstacles to the sharing of data, materials, code described in the work. HLC also ensure that the entire author group is fully aware of and in compliance with best practices in the discipline of publication. All authors read and approved the nal manuscript. Figure 1 Study ow chart.

Figure 2
Forest plot of 13-paired included studies.