Managing clustering effects and learning effects in the design and analysis of randomised surgical trials: A review of existing guidance

doi:10.21203/rs.3.rs-1403219/v1

PDF

Research Article

Managing clustering effects and learning effects in the design and analysis of randomised surgical trials: A review of existing guidance

https://doi.org/10.21203/rs.3.rs-1403219/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

The complexities associated with delivering randomised surgical trials, such as clustering effects, by centre or surgeon, and surgical learning, is well known. Despite this, approaches used to manage these complexities, and opinions on these, vary. Guidance documents have been developed to support clinical trial design and reporting. This work aimed to identify and examine existing guidance and consider their relevance to clustering effects and learning curves within surgical trials.

Methods

A review of existing guidelines, developed to inform the design and analysis of randomised controlled trials, is undertaken. Guidelines were identified using an electronic search, within the Equator Network, and by a targeted search those endorsed by leading UK funding bodies, regulators, and medical journals. Eligible documents were compared against pre-specified key criteria to identify gaps or inconsistencies in recommendations.

Results

Twenty-eight documents were eligible (12: Equator Network; 16 targeted search). Twice the number of guidance documents targeted design (n/N=20/28, 71%) than analysis (n/N=10/28, 36%). Managing clustering by centre through design was well documented. Clustering by surgeon had less coverage and contained some inconsistencies. Managing the surgical learning curve, or changes in delivery over time, through design was contained within several documents (n/N=8/28, 29%), of which one provided guidance on reporting this and restricted to early phase studies only. Methods to analyse clustering effects and learning were provided in five and four documents respectively. (N=28)

Conclusions

To our knowledge this is the first review as to the extent to which existing guidance for designing and analysing randomised surgical trials covers the management of clustering, by centre or surgeon, and the surgical learning curve. Twice the number of identified documents targeted design aspects than analysis. Most notably, no single document exists for use when designing these studies, which may lead to inconsistencies in practice. The development of a single document, with agreed principles to guide trial design and analysis across a range of realistic clinical scenarios, is needed.

Registration

Not applicable.

Trials

clinical trial

randomised controlled trial

complex intervention

surgical intervention

trial design

trial analysis

summary

review

clustering

learning

Randomised controlled trials (RCTs) are recognised as providing the highest level of evidence, second only to systematic reviews of such trials. [Evans, 2003] There are many practical and methodological difficulties that a medical researcher must overcome to deliver successful RCT. In trials with a surgical intervention, these difficulties are often magnified. [Cook. 2009; Craig, 2019; Ergina, 2009; McCulloch, 2009] Surgical interventions, delivered as an intervention or as a setting, consist of many interacting components - such as the procedure itself, surgeon expertise, and pre or postoperative care. [Ergina, 2009]

Patient outcomes often depend on the treatment provider delivering the intervention. Due to the nature of surgical interventions, RCTs within this field can be vulnerable to criticism if concerns over variability in treatment delivery are raised. Variability can arise between intervention providers (clustering) or due to change in delivery over time, often as a result of increased experience (learning). [ICH E9, 1999]. Therefore, when designing these trials, it is important to consider homogeneity of the treatment effect and therefore the potential existence and impact of both clustering and learning, by centre and surgeon. This should be done as early as possible during trial design to avoid issues arising that violate the validity of the trial results. [ICH E9, 1999]

The importance of managing these effects within these trials is well known, but the methods used to do so in practice, and opinions on these, varies. [Conroy, 2019; Conroy 2019; Conroy, 2020] Guidance documents exist to support clinical trial design and reporting, but the majority target generic aspects of clinical trials and originate specifically from medicinal trials. Therefore, whilst their relevance to all trials is indisputable, the extent to which they cover clustering and learning may be limited.

The aim of this review is to identify and examine existing guidance and consider their relevance to clustering effects and learning curves within surgical trials.

This work sought to include guidance documents developed to inform the design and analysis of randomised controlled trials (RCTs). Guidelines for inclusion in the cohort were identified by undertaking:

An electronic search within the Equator Network (http://www.equator-network.org), an online library containing a comprehensive searchable database of reporting guidelines, using each of the search terms “surgery” and “statistic”. Documents that provided guidance specific to non-randomised studies, aspects of trial methodology or medical specialties that were not applicable, or focussed on applicable medical specialties, such as surgery, with no statistical scope were excluded.
A targeted search of guidelines endorsed by leading UK funding bodies, regulators, and medical journals such that they covered aspects of trial design, analysis and reporting.

Because of the nature of the search, full texts of identified guidelines were obtained to determine eligibility. Documents that provided guidance such that RCTs and statistical aspects were covered within their scope were included and reasons for exclusion were recorded.

Key criteria relevant to design and analysis of surgical trials, or trials of complex interventions, were identified a priori, see Box 1. Eligible documents were compared against these to identify gaps or inconsistencies in recommendations. Guidelines for reporting the aspects of design and analysis were also assessed against these criteria. Specific methods within the guidelines related to analysing clustering or learning, at the centre or treatment provider level, were also collected. Documents were examined using NVivo qualitative data analysis software (QSR International Ply Ltd. Version 12, 2018).

Box 1: Key criteria to be considered within design and analysis

Design

• The appropriate trial design, such as an expertise-based design.

• Delivery of the intervention in terms of:

• The health professionals delivering treatment.

• The extent to which treatments are to be standardised.

• The potential for change in delivery over time.

• Adjusting the sample size.

• Balancing treatment within centres and treatment providers.

Analysis

• When the randomisation has been stratified.

• When analysing the primary outcome, such as adjustment.

• When there are multiple centres and/or treatment providers.

Identifying eligible documents

The search within the EQUATOR website identified 80 documents: 36 (45%) were identified using the search term “statistic” and 44 (55%) using the search term “surgery”. The search was conducted on 21st October 2021. Figure 1 presents the flowchart of eligibility, with reasons for exclusion where necessary. An additional 16 documents were manually identified from the targeted search (Funders: 2; Regulators: 6; Journals: 8, see Supplementary Material 1). There were no duplicates between the two searches leaving a total of 28 eligible documents for review. Supplementary Material 1 provides the list of included documents.

Designing a trial with clustering and learning

Choosing a trial design

Eleven out of 28 documents (39%) provided guidance relating to trial design. See D1 in Supplementary Table 1.

The options of trial design depend on the unit of randomisation and the intervention of interest. The key aspects of relevant designs are briefly summarised here. Many design options, and associated limitations, were discussed and no single document provided a single comprehensive summary.

In individually randomised trials, patients are the unit of randomisation. [Craig, 2019] When conducting these trials in surgery, differential expertise between the treatments being investigated can raise issues that can be alleviated by defining eligibility criteria for centres and surgeons, such as years in practice or number of interventions performed previously. [Boutron, 2008; National Institute for Health Research website, 2016] However, applying criteria that is too strict may reduce the generalisability of trial results. [Zwarenstein, 2008] Instead, a statistical analysis of inter-rater reliability, between individual centres and surgeons, can provide understanding of any impact due to expertise. This type of analysis can be useful when considering rolling out the interventions into routine healthcare, see Analysing a trial with clustering and learning. [National Institute for Health Research website, 2016]

In cluster randomised trials, groups of patients are the unit of randomisation. These designs are less common and are generally less efficient than individually randomised studies. They require more surgeons and introduce the potential for the treatment comparison to be confounded by the delivery, despite inflating the sample size to account for the intraclass correlation coefficient (ICC). [Craig, 2019; Ergina, 2009; Campbell, 2012]

Expertise-based designs are a half-way house between individual and cluster randomised trials. Patients are individually randomised to surgeon, who treats all patients with a single intervention. This can be the surgeon’s preferred technique or are an intrinsic feature in trials comparing interventions delivered by different specialties. [Boutron, 2008; Ergina, 2009] This design has the same limitations as cluster trials, and when a surgeon is only performing their preferred technique, shared waiting lists [Ergina, 2009] and understanding how the treatment can be rolled out into routine healthcare can be a challenge. Resultantly, this design is relatively uncommon. [Conroy, 2019; Conroy, 2019]

Tracker designs, proposed by Ergina et al, where new or evolving interventions can theoretically be developed within a single randomised study, and the incremental changes to the intervention tracked within the analysis, would be very challenging in practice. [Ergina, 2009]

Considering who will deliver the intervention

Thirteen out of 28 documents (46%) discussed the importance of deciding who will deliver the intervention. See D2 in Supplementary Table 1.

Some variation in delivery, in part, will depend on the skill and training of those delivering the intervention. [Boutron, 2008; Ergina, 2009; Jackson, 2010] As such, the selection of centres and treatment providers was a critical element of design discussed by a number of guidance documents. [Campbell, 2012; ICH E9, 1999; ICH E6, 1996; MHRA, 2021] Any eligibility criteria for participating centres and treatment providers, and a description such as the degree to which they are typical, should be reported. [Boutron, 2008; Zwarenstein, 2008; Boutron, 2017]

Suggested selection criteria for centres related to caseload for the procedure under investigation and, likewise, ensuring sufficient numbers of the target population and facilities to deliver the trial. [Boutron, 2008; MHRA, 2021]

No guidelines provided advice on selecting treatment providers. Treatment providers could be a limited group or all professionals offering the intervention. [Elias, 2019] If a limited group, guidance on selecting centres, and reporting requirements, may be looked upon as a proxy for triallists when deciding how to select treatment provider, for example caseload and ensuring specific qualifications. [Boutron, 2008; MHRA, 2021; Boutron, 2017]

The results of the main trial should report on the number of centres and treatment providers performing each intervention. [Boutron, 2017]

Ensuring that the intervention is standardised

Fifteen out of 28 documents (54%) discussed the importance of standardising the intervention. See D3 in Supplementary Table 1.

Variation in delivery can be reduced by standardising all, or aspects of, the intervention of interest. Limiting variation in treatment delivery may be more desirable in an efficacy trial than a pragmatic, effectiveness study. [Craig, 2019; McCulloch, 2009] In pragmatic trials. standardisation might consist of simply informing treatment providers to perform the treatment as usual. [Boutron, 2008] Regardless of the stage, trial delivery should be similar at all centres [ICH E9, 1999] and designed such that a clear description of the procedures performed can be provided. [Zwarenstein, 2008; Vanhie, 2016] Investigator meetings to prepare investigators and standardise performance were suggested by one document. [ICH E8, 1998]

Monitoring treatment adherence was an important aspect across documents. [Boutron, 2008; ICH E9, 1999; McCulloch, 2009; ICH E8, 1998; ICH E3, 1995] Suggested methods included reviewing case report forms, videotapes and audiotapes, extending to decertifying and excluding surgeons not submitting a videotape rated acceptable by an independent committee. [Boutron, 2008]

Reporting in depth details of the intervention, and comparator, was required by a number of documents. Aspects required included technical procedures, full details on preoperative, intraoperative and postoperative care and the extent to which delivery were permitted to vary between participants, treatment providers and centres. [Boutron, 2008; Zwarenstein, 2008; ICH E3, 1995]

Anticipating changes over time

Eight out of 28 documents (29%) discussed considering changes in delivery of the intervention over time. See D4 in Supplementary Table 1.

Delivery may still vary irrespective of training, experience and other steps to enforce standardisation. The amount of variation will depend on the stage and technicality of intervention development. [Craig, 2019; Boutron, 2008; McCulloch, 2009; Bilbro, 2021] An important aspect of surgical evaluation across the guidelines was that delivery may change over time for pragmatic reasons, changes in external factors, or as a result of expertise developing during the study. [Craig, 2019; Ergina, 2009; McCulloch, 2009]

Expertise can develop over a very long time and so requiring a set expertise level can slow the delivery of surgical trials. [Ergina, 2009] Some guidelines discussed evaluating the learning curve within the trial, [McCulloch, 2009] and highlighted this was particularly important in earlier phase trials. [Bilbro, 2021] In trials comparing more established techniques, the statistical advantages and gain in ‘internal validity’ need to be considered against the loss of generalisability or ‘external validity’ of applying too much emphasis on the learning curve. [Craig, 2019]

Reporting learning curve assessment results was required by one document but this was limited to early phase studies. [Bilbro, 2021]

Estimating the sample size

Eight out of 28 documents (29%) discussed sample size. See D5 in Supplementary Table 1.

A number of guidance documents highlighted the impact of failing to reduce variation within trial arms by standardising the intervention on the sample size and power calculation, where typical estimates assume that differences between the treatments across centres, or treatment provider, are unbiased estimates of the same quantity. [Craig, 2019; ICH E9, 1999] In the presence of multilevel data structures, where variability in individual level outcomes can reflect higher level processes, calculations are more complicated. [Jackson, 2010; ICH E9, 1999; Cook, 2004] To avoid associated imprecision in results, the sample size should adjust for any clustering effects as estimated by the intraclass correlation coefficient (ICC) and this should be reported in the main results paper. [Boutron, 2008; Boutron, 2017] Conversely, two documents that discussed sample size did not comment on adjusting for clusters. [National Institute for Health Research website, 2016; MHRA, 2021]

Ensuring balance of treatment within centre and treatment provider

Six out of 28 documents (21%) discussed ensuring that treatment allocations are equally distributed within centre. See D6 in Supplementary Table 1.

Balancing treatment groups with respect to prognostic factors enhances trial credibility. [MHRA, 2021; EMA, 2021] Ensuring balancing of patients within centre was highlighted as important within many of the guidance documents, [ICH E9, 1999; MHRA, 2021; EMA, 2021] and similar reasoning would lead surgical triallists to extend this to treatment provider which was not discussed within any document.

Balance can be achieved by stratifying the randomisation and stratifying by centre was a common topic, particularly when centre is expected to be confounded with other prognostic factors. [ICH E9, 1999; MHRA, 2021; EMA, 2021] When there are too few patients per centre, stratifying by a larger unit, such as country or region, may be warranted. [EMA, 2021] Despite stratifying by treatment provider not being specifically addressed within the documents, in some circumstances it may be desirable to stratify for more than just both centre and treatment provider, or treatment provider alone, where numbers allow. [EMA, 2021] The use of more than two stratification factors is rarely necessary. [ICH E9, 1999]

Analysing a trial with clustering and learning

When the randomisation was stratified

Two out of 28 documents (18%) provided guidance on adjusting the analysis following stratification. See A1 in Supplementary Table 1.

Stratifying randomisation and subsequently adjusting the analysis, are complementary methods of accounting for prognostic factors, unless the stratification factor was chosen for administrative reasons only. [ICH E9, 1999; EMA, 2021]

Two documents discussed the issue of adjusting for too many, or too small, strata in the analysis, for which there is no best solution. [ICH E9, 1999; EMA, 2021] When included in the randomisation scheme, ignoring centres or adjusting for a large number of small centres might lead to unreliable estimates of the treatment effect and p-values. [EMA, 2021] At best, using an unadjusted analysis should be supported by sensitivity analyses that indicate trial conclusions are not affected because of this. [EMA, 2021] As above, the statistical justifications for including centre could be considered to also include treatment provider in surgical trials but no guidance required specifically.

When analysing the primary outcome

Two out of 28 documents (18%) provided guidance on adjusting the primary outcome analysis. See A2 in Supplementary Table 1.

Unexplained differences between treatments, for example between adjusted and unadjusted analyses, can jeopardise the trial results. [EMA, 2021] For this reason, when the primary outcome is expected to be influenced by centre or treatment provider, an adjustment should be planned. When the potential value of an adjustment is in doubt, such as little existing prior knowledge, the primary analysis should be unadjusted analysis, supported by an adjusted analysis. [ICH E9, 1999; EMA, 2021] In general, larger datasets generally support more factors than smaller ones and results based on simpler models are generally numerically stable, the assumptions underpinning the statistical model easier to validate and improves generalisability. [EMA, 2021]

Analysing multi-centre trials

Six out of 28 documents (21%) provided guidance on analysing multi-centre trials. See A3 in Supplementary Table 1.

Investigations into heterogeneity of the main treatment effect across centre and/or treatment provider were covered by a number of documents. [Boutron, 2008; ICH E9, 1999; McCulloch, 2009; ICH E3, 1995; Bilbro, 2021] Further, the main trial publication should report methods to adjust for, and results into, clustering by centre or treatment provider. [Boutron, 2008; Boutron, 2017] These investigations are critical when a positive treatment effect is found and there are appreciable numbers of subjects per centre. [ICH E9, 1999] In the simplest multi-centre trial, a single investigator recruits and is responsible for all patients within a single hospital, such that centre is identified uniquely by hospital. When the definition of centre is ambiguous, such as a single investigator recruits from several hospitals or a clinical team recruits from numerous clinics, the protocol should provide a definition. [ICH E9, 1999; ICH E3, 1995]

Quantitative approaches may comprise graphical display of the results of individual centres, such as forest plots, or analytical methods, such as a significance test although this generally has low power. [ICH E9, 1999] One stated that investigations use a model which allows for centre differences but no interaction terms, [ICH E9, 1999] Fixed or mixed effects models can be used, although mixed models are especially relevant when there is a large number of centres. [ICH E9, 1999; ICH E3, 1995]

Methods for investigating the learning curve

Four out of 28 documents (14%) provided guidance on analysing the learning curve within centre and/or treatment provider. See A4 in Supplementary Table 1.

Reporting of continuous quality control measures can be useful for all phases of trial, particularly early phase surgical trials. [McCulloch, 2009; Bilbro, 2021] Time series and longitudinal models or multilevel models can be used to analyse long and short sequences of data respectively. [Craig, 2019; Jackson, 2010] Simpler exploratory methods such as cusum plots enable centres or surgeons to be compared against themselves which can be preferable to surgeons. [McCulloch, 2009; Bilbro, 2021]

Method for investigating clustering

Five out of 28 documents (18%) provided guidance on investigating clustering due to centre and/or treatment provider. See A5 in Supplementary Table 1.

Hierarchically structured data, such as patients within surgeon, can be analysed using multilevel models or generalised estimating equations (GEE). [Craig, 2019; Boutron, 2017] Multilevel models are subject-specific models whereas GEEs are population average. For multilevel models: fixed, random or mixed effects can be specified to account for clustering [Boutron, 2017] and different types of these models allow for flexible data structures. [Jackson, 2010]

For ordinary linear models, the treatment effect estimate is likely to be similar but not necessarily identical for adjusted and unadjusted models. Adjusted analyses are more efficient, and so a less significant result for unadjusted should not be a concern. For generalised linear or non-linear models, adjusted and unadjusted treatment effects may not have the same interpretation and may provide different results. [EMA, 2021]

Triallists should consider the impact of clustering and learning when designing and analysing randomised surgical trials. Considerations should be incorporated into reporting to aid interpretation and applicability of trial results. This investigation is the first review as to the extent that existing guidance covers these important effects. Existing guidance documents are identified and summarised, with a focus on aspects relating to clustering effects and the learning curve and their application to surgical trials. Not all documents were written specifically for surgery, yet all contain aspects that can be applied to surgery, for example the role of centre in delivery of treatments in drug trials is not dissimilar to the role of surgeon in delivering a surgical trial. Twice the number of identified documents targeted design aspects than analysis. Whilst a good analysis cannot rescue a poor design, and this may have led to a larger focus on design on guidance for triallists, there is a notable dearth of analysis guidance available that requires addressing. In addition, there is also scope for guidance on study conduct.

Clustering, at the centre level, was well covered within the design, analysis and reporting guidance. However, the were inconsistencies with regards to the treatment provider coverage. For example, reporting required eligibility of treatment provider be covered, yet no guidance on the design or analysis covered this. [Boutron, 2008; Boutron, 2017] However, this may be due to the original guidance largely not being written specifically for surgery, or indeed complex interventions, where these effects may be more prominent. [Conroy, 2019] The role of centre within conventional drug trials could be extended to provide guidance on the role of treatment provider in surgery trials. [Craig, 2019; ICH E9, 1999; MHRA. 2021] Methods to monitor the quality of delivery of the trial intervention through study conduct has been considered, recommending that stablished protocols that determine prohibited, mandated and flexible intervention components and monitoring adherence are developed. [Blencowe. 2016] Yet a specific guidance document that covers the design and analysis of randomised surgical trials, or intervention trials, could address the discrepancies identified within this review to improve quality of understanding and awareness of these issues. [Conroy, 2019; Conroy, 2020].

A number of guidance documents acknowledged the importance of the surgical learning curve, or delivery changing over time, within design and analysis, particularly in early phase surgical trials or when the interventions differ in their technicality [Craig, 2019; Boutron, 2008; Ergina, 2009; McCulloch, 2009; Bilbro, 2021] Yet there was little coverage within reporting standards to reflect this, with the surgical learning curve analysis only necessary in early phase, and not necessarily randomised, trials [Bilbro, 2021] and broader RCT reporting guidelines only requiring differential expertise be addressed in the discussion. [Boutron, 2008] Lack of clear standards, and guidance that is too broad in scope, may lead to reporting how delivery of intervention changes over time, despite its importance, being generally under-recognised in the literature. [Conroy, 2019]

When reading this review, it is important to consider its limitations. First, country specific guidance beyond the UK, such as United States Food and Drug Administration, were not included. However, international documents that are applicable to other countries, including the UK, were obtained, such as ICH which are followed globally and EMA which are adopted within Europe. Second, only four guidance documents, developed by the same research group, were written specifically for surgery and not written specifically for RCTs, which may explain the lack of specific coverage of the surgeon in the wider set. [Ergina, 2009; McCulloch, 2009; Bilbro, 2021; Barkun, 2009] Third, very little of the guidance documents covered statistical aspects, leaving a triallist to extend the centre-drug connection to surgeon-intervention using existing guidelines. [ICH E9, 1999; EMA, 2021] A statistical guidance document that covers randomised surgical trials in more depth would help triallists, in particular statisticians, and the IDEAL framework provides a good basis for this development. [Ergina, 2009; McCulloch, 2009; Bilbro, 2021; Barkun, 2009]

This is the first review, to our knowledge, to explore coverage of guidance for managing clustering effects and the surgical learning curve within the design and analysis of randomised surgical trials. Twice the number of identified documents target design aspects than analysis. Furthermore, no single and complete guidance document exists that covers aspects of learning and clustering leaving triallists to have to access multiple documents to gain full understanding of these considerations. The development of such a document would encourage better consistency between triallists, improve awareness of these methodological challenges and support the use of optimal methods.

GEE	Generalised estimating equations
ICC	Intraclass correlation coefficient
RCT	Randomised controlled trial

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

Datasets used and/or analysed during the current study, that are not already included in this published article, are available from the corresponding author on reasonable request.

Competing interests

CG, JAC and JMB co-authored some of the guidance documents included within this review. The authors declare that they have no other competing interests.

Funding

This research was funded by National Institute for Health Research (NIHR) Doctoral Fellowship Programme (DRF-2015-08-082). EJC is funded through this fellowship programme.

Authors’ contributions

EJC participated in the study design, developed the data extraction form, drafted the manuscript, and extracted and analysed the data. CG participated in the study design, developed the data extraction form, and drafted the manuscript. GB, JMB and JAC participated in the study design and contributed to manuscript development. All authors read and approved the final manuscript.

Acknowledgements

None.

Department of health disclaimer

The views expressed are those of the authors and not necessarily of the National Health Service, the National Institute for Health Research (NIHR) or the Department of Health and Social Care.

Barkun JS, Aronson JK, Feldman LS, Maddern GJ, Strasberg SM, Balliol C, et al. Evaluation and stages of surgical innovations. Lancet. 2009;374(9695):1089–96.
Bilbro NA, Hirst A, Paez A, Vasey B, Pufulete M, Sedrakyan A, et al. The IDEAL Reporting Guidelines: A Delphi Consensus Statement Stage Specific Recommendations for Reporting the Evaluation of Surgical Innovation. Ann Surg. 2021;273(1):82–5.
Blencowe NS, Mills N, Cook JA, Donovan JL, Rogers CA, Whiting P, Blazeby JM. Standardizing and monitoring the delivery of surgical interventions in randomized clinical trials. Br J Surg. 2016 Sep;103(10):1377–84. doi:10.1002/bjs.10254. Epub 2016 Jul 27. PMID: 27462835; PMCID: PMC5132147.
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Grp C. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Ann Intern Med. 2008;148(4):295–309.
Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P, Group CN. CONSORT Statement for Randomized Trials of Nonpharmacologic Treatments: A 2017 Update and a CONSORT Extension for Nonpharmacologic Trial Abstracts. Ann Intern Med. 2017;167(1):40–7.
Campbell MK, Piaggio G, Elbourne DR, Altman DG, Group C. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661.
Committee for Medicinal Products for Human Use (CHMP). Guideline on adjustment for baseline covariates in clinical trials2015 30th November 2021. Available from: https://www.ema.europa.eu/en/adjustment-baseline-covariates-clinical-trials.
Conroy EJ, Rosala-Hallas A, Blazeby JM, Burnside G, Cook JA, Gamble C. Randomized trials involving surgery did not routinely report considerations of learning and clustering effects. J Clin Epidemiol. 2019;107:27–35.
Conroy EJ, Rosala-Hallas A, Blazeby JM, Burnside G, Cook JA, Gamble C. Funders improved the management of learning and clustering effects through design and analysis of randomized trials involving surgery. J Clin Epidemiol. 2019;113:28–35.
Conroy EJ, Blazeby JM, Burnside G, Cook JA, Gamble C. Managing clustering effects and learning effects in the design and analysis of multicentre randomised trials: a survey to establish current practice. Trials. 2020;21(1):433.
Cook JA, Ramsay CR, Fayers P. Statistical evaluation of learning curve effec ts in surgical trials. Clin Trials. 2004;1.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions2019; 2019(January 23rd). Available from: https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/.
Elias KM, Stone AB, McGinigle K, Tankou JI, Scott MJ, Fawcett WJ, et al. The Reporting on ERAS Compliance, Outcomes, and Elements Research (RECOvER) Checklist: A Joint Statement by the ERAS((R)) and ERAS((R)) USA Societies. World J Surg. 2019;43(1):1–8.
Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, et al. Challenges in evaluating surgical innovation. Lancet. 2009;374.
Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs. 2003;12(1):77–84.
ICH Expert Working Group. ICH Harmonised Tripartite Guideline: Structure and Content of Clinical Study Reports E3. ICH Harmonsation for better health web site: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use.; 1995 [Available from: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E3/E3_Guideline.pdf.
ICH Expert Working Group. ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice E6(R1). ICH Harmonisation for better health web site: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use.; 1996 [Available from: https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R1_Guideline.pdf.
ICH Expert Working Group. ICH Harmonised Tripartite Guidelines: General Considerations for Clinical Trials E8.1998. Available from: https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-8-general-considerations-clinical-trials-step-5_en.pdf.
Jackson DL. Reporting results of latent growth modeling and multilevel modeling analyses: some recommendations for rehabilitation psychology. Rehabil Psychol. 2010;55(3):272–85.
McCulloch P, Altman DG, Campbell B, Flum DR, Glasziou P, Marshall J, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374.
Medicines & Healthcare products Regulatory Agency. Clinical investigations of medical devices – statistical considerations2021. Available from: Statistical_considerations_clinical_investigations_-_May_2021.pdf publishing.service.gov.uk.
National Institute for Health Research. Planning a Randomised Controlled Trial (RTC) – Points to Consider2016. Available from: https://www.ct-toolkit.ac.uk/routemap/trial-planning-and-design/downloads/planning-a-randomised-controlled-trial.pdf/.
Vanhie A, Meuleman C, Tomassetti C, Timmerman D, D'Hoore A, Wolthuis A, et al. Consensus on Recording Deep Endometriosis Surgery: the CORDES statement. Hum Reprod. 2016;31(6):1219–23.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. Brit Med J. 2008;337.

PDF

Reviews received at journal
05 Apr, 2022
Reviewers invited by journal
05 Apr, 2022
Editor assigned by journal
05 Mar, 2022
First submitted to journal
27 Feb, 2022

You are reading this latest preprint version

Managing clustering effects and learning effects in the design and analysis of randomised surgical trials: A review of existing guidance

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Identifying eligible documents

Designing a trial with clustering and learning

Choosing a trial design

Considering who will deliver the intervention

Ensuring that the intervention is standardised

Anticipating changes over time

Estimating the sample size

Ensuring balance of treatment within centre and treatment provider

Analysing a trial with clustering and learning

When the randomisation was stratified

When analysing the primary outcome

Analysing multi-centre trials

Methods for investigating the learning curve

Method for investigating clustering

Discussion

Conclusions

List Of Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1