An emerging methodology known as the Multiphase Optimization Strategy (MOST), which was inspired by engineering frameworks and guides research questions related to identifying the optimal version of an intervention, is receiving increasing attention in the healthcare field. The MOST framework includes three phases: preparation of a conceptual model with identified intervention components that impact the intervention effectiveness, optimization of the intervention with a trial that evaluates the performance of the individual intervention components, and evaluation of the optimized intervention with a randomized controlled trial (RCT). Unlike the traditional RCT framework which compares a treatment group that receives an intervention package to a control group, the MOST framework tests the anticipated “active ingredients” (i.e., intervention components), thus providing results on the most effective, efficient, and scalable form of an intervention [1, 2].
MOST optimization trials utilize factorial experimental design [3, 4, 5] because they can test multiple factors (i.e., intervention components) simultaneously, using the same participants while maintaining satisfactory statistical power [6]. For example, a factorial design with two intervention components with two levels each yields four cells (i.e., 2 x 2 = 4), each representing a group of participants assigned to a study condition that receives a unique combination of intervention component levels. As the number of intervention components in the factorial design increases, the number of cells grows exponentially (i.e., four components with two levels each requires 2 x 2 x 2 x2 = 16 cells). Because participants in factorial experiments are independently assigned to a level on each factor and factors are analyzed separately for main effects, statistical power will generally be equivalent to a single-factor RCT that has the same number of study arms as the factorial design’s number of levels within each factor.
Despite the benefits that factorial designs offer with regard to sample size and statistical power, they also present complexity and challenges for subject allocation, especially when the number of cells is large. Consensus guidelines for reporting randomized trials (i.e., Consolidated Standards of Reporting Trials (CONSORT); [7]) describe a range of acceptable methods for allocation of participants to study cells and suggest that three criteria are important for determining which method to use. First, participant allocation should result in balanced sample sizes across study conditions to maximize statistical power [7, 8, 9]. Second, participant allocation should result in study conditions that are equivalent with respect to covariates that are expected to impact intervention outcomes (i.e., equivalent groups; [10]). Third, participant allocation should be completely unpredictable to both study staff and to participants so as to ensure that measured and unmeasured participant characteristics, and selection biases in general, do not influence participants’ assignment to conditions. Given the large number of cells in factorial experiments, balanced sample sizes and equivalent groups are especially important, yet may be difficult to achieve. Finally, we suggest additional criteria that are common to many practical decisions: cost and complexity including resource utilization. Some allocation procedures can be readily implemented using a range of accessible methods and software, whereas other methods may require coding or specialized software that must then be incorporated into work flow. The four outcomes of interest in the present study are these four criteria for subject assignment methods: balance of sample size, equivalence of groups, unpredictability, and complexity.
The CONSORT statement [7] classifies the range of acceptable assignment methods into three categories: Simple randomization, which includes the use of random number tables, computerized random number generators, or even a coin toss. Restricted randomization, which involves combining random assignment with additional strategies to improve balance and equivalence across cells. For example, assigning participants in blocks that are the same size as (or a multiple of) the number of cells promotes balance across conditions in sample size [7, 9]. Stratification defines subsets of participants within which random assignment with blocking occurs [7], thus promoting equivalence in the baseline characteristics used to define the strata [11]. Minimization involves assigning each participant to the condition that minimizes differences in pre-specified covariates across study cells. Minimization can be specified to include random assignment, but the fundamental procedure is deterministic [7].
As expected, each of these subject allocation methods has strengths and weaknesses with regard to balance, equivalence, unpredictability, and complexity (see Table 1). With regard to predictability, simple randomization is the most unpredictable on a theoretical level and is therefore best for reducing the threats of selection bias [9, 12, 13]. In comparison, restricted randomization maintains the use of random assignment but typically does so within blocks of participants. If block size is known to study investigators, assignment becomes increasingly predictable as cumulative enrollment reaches numbers equivalent to multiples of the number of study conditions [7, 14]. While introducing random variation in block size (i.e., permuted blocks) can mitigate this problem [7], the benefit of doing so with respect to balance declines as the number of study cells increases—as is often the case for factorial experiments. Thus, the degree of predictability of restricted randomization depends on the exact procedure used and the study design to which it is applied. Likewise, minimization procedures are a purely deterministic method that could be predicted given access to covariate values for all participants as well as the algorithm by which these values result in assignment. However, such predictions are often very complex and difficult to keep track of in one’s mind. Moreover, elements of random assignment can also be introduced to further mitigate this problem. For minimization, predictability again depends on the exact procedure used and the study design to which it is applied.
Table 1. Comparison of allocation methods
|
Unpredictability
|
Balanced sample sizes across conditions
|
Equivalent baseline characteristics across conditions
|
Cost & complexity
|
Simple randomization
|
Best; random assignment prevents selection bias
|
Worst; likely to result in differences across cells
|
Worst; likely to result in differences across cells
|
Best; simple to implement
|
Stratified randomization with blocking
|
Very good; random assignment prevents selection bias, but blocking introduces some predictability
|
Very good; blocking improves balance, but this is mitigated by stratification
|
Good; stratification improves equivalence on specific variables
|
Very good; more complex, but solutions are widely available
|
Minimization
|
Good; complexity reduces predictability, but randomness can also be included
|
Very good; should promote balance, depending on algorithm
|
Best; promotes equivalence on a large number of variables
|
Good; can be implemented in a range of available software, but requires coding
|
When seeking balanced sample sizes and groups that are equivalent with respect to baseline variables across study conditions, simple randomization is expected to perform the most poorly [9]. The credibility of factorial experiments can be significantly compromised by simple randomization because of the compounded problem of yielding cells that are imbalanced with respect to sample size and non-equivalent with respect to key covariates [7]. Thus, many researchers favor restricted randomization involving blocking with stratification for factorial designs as well as single-factor RCTs; indeed, stratification with blocking is the dominant participant assignment procedure in randomized controlled clinical trials [14]. However, stratification with blocking is not a perfect method. In addition to the predictability of assignment that can occur when block size is equivalent to the number of study conditions, stratification is only feasible for two or three variables at most [11]. It is therefore limited in the number of variables on which it can promote equivalence.
Minimization procedures can ensure that conditions have balanced sample sizes and equivalent baseline characteristics for a large number of variables, even for studies with small sample sizes and/or many treatment conditions, across all stages of an experiment. Some therefore argue that minimization procedures are not only acceptable, but a superior alternative to simple or restricted randomization [15]. In minimization assignment, the first patient is assigned at random, and each following participant is assigned to the condition that minimizes differences across study conditions with respect to selected factors. Although minimization is a deterministic method, assignment becomes less predictable to researchers as more variables are added and the minimization algorithm becomes more complex [16]. Moreover, researchers may incorporate randomization into their minimization scheme by using a weighted probability that favors, but does not determine, assignment to the condition that minimizes imbalances [7], which appears to be an effective mechanism to prevent the risk of minimization assignment from being "deterministic" [17].
Finally, assignment strategies differ with respect to cost and complexity. In this regard, simple randomization is arguably best as a wide range of software and even a coin or die can be sufficient. Restricted randomization is a close second because it is included in a range of software packages used by researchers, including RedCap. While simple conceptually, minimization procedures are currently the most difficult to implement, requiring specialized coding in software packages like Excel, Stata, or R. Minimization algorithms must be individualized based on covariates of interest in particular studies. Once the minimization program is written, study staff must obtain sufficient covariate data (e.g., age, ethic identity, co-morbid conditions) to run the program. The program will likely be stand-alone as no minimization programs are currently integrated into other commonly used research study management systems such as RedCap or StudyTrax. Overall, minimization is the most complex because it requires additional skills, staff time, and resources.
Aims of the Current Study
Given the relative strengths and weakness of simple randomization, restricted randomization, and minimization procedures for participant assignment with respect to a given study design (e.g., number of cells, sample size) and hypotheses (e.g., number of conditions of interest, importance of testing interactions), it is critical that researchers make deliberate, informed choices about their participant assignment procedures for their unique studies, particularly in the context of MOST frameworks and other factorial designs. The primary aim of this paper is to present a case study of how assignment procedures can be directly compared by conducting and comparing simulations of each assignment method on a locally-collected dataset.