We aimed to describe the characteristics related to the design of a cohort of self-labelled pRCTs in MT according to their pragmatic attitude. We obtained three main findings. First, MT pRCTs are not becoming more common, despite the increased attention on this type of design. Second, our findings revealed that pRCTs in MT have a moderately pragmatic attitude. However, the label 'pragmatic' could be questioned in those trials using highly explanatory design features like blinding, unicentric setting, or using a placebo as a control intervention. Lastly, the authors of our sample of pRCTs did not adequately justify why the trial aimed to be pragmatic and which domains most contributed to this pragmatism.
Although research in the MT field has increased in the last decades [75–77], pRCTs are not so common. Only 37 papers assessing an MT intervention were identified as self-labelled pragmatic from inception to 2022. Our results indicate that the number of MT pRCTs has not increased in recent years, and the PRECIS-2 mean score has also not increased over time. These results are contrary to studies assessing pragmatism in critical care [78], pain [5] and cardiovascular interventions [79], wherein the mean PRECIS score increased over time. Only Domain 5 (flexibility intervention) increased after 2018, primarily due to the significant decrease in the mean PRECIS scores between 2012 and 2018. It might be interesting to explore the knowledge about non-pharmacological pRCTS among researchers to understand this lack of increase over time. It might be interesting to assess the necessity to include specific training on this topic in the academic stages.
The mean score of our sample in the retrospective PRECIS-2 tool assessment was 3.5 (SD = 0.6), indicating a moderate level of pragmatism of MT pRCTs. Those are similar ratings to a recent systematic review of nursing interventions and those found in pain interventions and other fields [5, 9, 10, 30, 35, 78, 79]. The moderate level of pragmatism found in our review was highly influenced by the domains “flexibility intervention”, “follow-up”, “outcome measure”, and “primary analysis”. Those domains were rated as pragmatic (4 or 5 on the Likert scale) in more than 60% of our assessed trials. Using an intention to treat analysis is a trait highly pragmatic, and most of the reviews assessing the pragmatism of trials in other fields found this item as the most pragmatic in their PRECIS-2 tool assessment [5, 35, 78, 79].
Considering the extensive debate about the appropriateness of self-labelled pragmatic trials using explanatory features such as blinding, unicentric setting and use of placebo controls [12, 75–78], we might exclude almost half of our sample (19/37). However, as there is no consensus on that matter, we decided to assess the whole sample and to further contribute to the debate as other authors did in other fields [5, 9, 30, 35].
Using a placebo as a comparator is considered a very explanatory feature. Some authors argue that it is hardly part of real-world care, and its use suggests an explanatory nature of the trial [12, 80–82]. PRECIS-2 authors defend that there are situations in which a trial could include a placebo control and still provide useful information to decision-making (e.g. to mitigate the subjectiveness of patient-reported outcomes in a comparative effectiveness trial of two accepted treatments) [7, 83]. We wonder if including an appropriate placebo/sham comparator could be interesting for understanding the real effect of the intervention, which is difficult to achieve in an efficacy study under non-real-world conditions (where all the contextual factors part of the patient-therapist encounter, are not present [84–87]). In our sample, only one study used a placebo control, and another included a placebo control in a 3-armed trial. In the latter, Mafetoni et al. showed that although the effect of the intervention was statistically larger than that in the control and placebo groups, the placebo group showed a larger effect than the control group [57]. Non-pharmacological interventions do not follow the same developing processes as pharmacological interventions do. This fact entails that effectiveness trials are often performed to assess interventions without performing previous efficacy trials. Further research should be conducted to contribute to this rich debate about using a placebo in pRCTs in the MT field [12, 82, 88].
Another controversial point is the use of single centres, as there is a debate about if they should be considered pragmatic because of the arguably limited generalisability of their results [80, 82, 83, 88]. In our sample, 25% of the reports were performed in a single-centre setting.
In terms of using a design tool to guide where the trial falls in the explanatory-pragmatic continuum, only one trial reported using the PRECIS-2 tool [50]. It is worth mentioning that this trial scored higher on average (4.1) than the rest. This limited reporting of using a design tool was also noted by other authors [35, 78, 79].
Recently, it has been pointed out that, when assessing pragmatic attitude in a trial, special attention must be paid to detecting its intention besides awarding a quantitative score [4, 80]. By knowing the trial's intention, some domains can be expected to be more pragmatic than others. Some authors stated that the pragmatic attitude is domain-specific and that PRECIS-2 ratings require context [5]. In fact, one of the fundamental principles of external validity is that context matters [3, 89]; it can help to understand the trial's attitude better, avoiding dichotomisation into pragmatic and explanatory [4]. In our sample, less than 20% of the trials reported that the aim was to investigate an intervention under real-world conditions and to make clinical decisions about its effectiveness, thus not identifying an intention aligned with a pragmatic attitude. So almost 80% of our assessed trials were labelling the trial as pragmatic for unknown reasons but not because of their pragmatic attitude.
Likewise, Janiaud et al. assessed the pragmatic attitude of self-labelled trials in different fields, founding that 45% of the authors did not adequately justify or discuss the claimed pragmatism [9].
Both efficacy and effectiveness trials are necessary but address different research questions [29]. A potential question is whether there are already enough efficacy trials to explore further an intervention's effects in a more world-real setting. The authors should be expected to screen previous literature to appraise what is already known about a given intervention before attempting a pRCTs. Ultimately, a sound background should be provided alongside the rationale for attempting a pragmatic trial [90]. Such rationale might include existing mechanistic experiments and placebo- or sham-controlled efficacy trials, suggesting that the intervention has the desired physiological effects and efficacy [91]. Instead, in the 22% of the trials we reviewed, the authors described neither the proof-of-concept of the intervention nor the state of previous studies addressing related research questions (rationale of the intervention). Furthermore, when the rationale was given, it was mainly constructed from comparative effectiveness trials. These results align with a similar review assessing self-labelled trials' pragmatic attitudes [30]. It found that 71.3% of the sample did not justify why a pragmatic trial was selected instead of an explanatory one [30]. This should be considered fundamental when designing a pRCT [1].
Although some authors declared that pRCTs might be an option for non-regulated interventions in early and late development trials [1, 10], we argue that researchers need to appraise the research field as a whole and determine whether there is sufficient evidence of the efficacy of a given intervention before assessing it in a real clinical environment. When efficacy trials have not sufficiently demonstrated the effects of an intervention, spending resources on a more explanatory design might be preferable to pRCT [91].
Our review has several potential limitations. Although the reviewer team included experts in the MT field and expert methodologists, providing a multidisciplinary approach to data extraction, this might introduce some heterogeneity to the data extraction process that could have affected the results. Also, a deep knowledge of each intervention assessed is required to assess the pragmatism of a given trial properly. Other authors have reported similar difficulties [4, 5, 10, 92]. To mitigate the impact of these limitations, we piloted the extraction data form and extensively discussed the ratings beforehand. Also, we paired all reviewers with the IP to discuss the scores and minimise subjectivity. Although a third party was not required to solve discrepancies, frequent discussions between reviewers were needed to conclude a reliable score. Finally, the small sample in this review might not be considered relevant and may not represent all the MT modalities. However, it included all pRCTs published in the MT field from inception to January 2022.