An emerging methodology known as the Multiphase Optimization Strategy (MOST), which was inspired by engineering frameworks and guides research questions related to identifying the optimal version of an intervention, is receiving increasing attention in the healthcare field. The MOST framework includes three phases: preparation of a conceptual model with identified intervention components that impact the intervention effectiveness, optimization of the intervention with a trial that evaluates the performance of the individual intervention components, and evaluation of the optimized intervention with a randomized controlled trial (RCT). Unlike the traditional RCT framework which compares a treatment group that receives an intervention package to a control group, the MOST framework tests the anticipated “active ingredients” (i.e., intervention components), thus providing results on the most effective, efficient, and scalable form of an intervention.1,2
MOST optimization trials utilize factorial experimental design3–5 because they can test multiple factors (i.e., intervention components or delivery strategies) simultaneously, using the same participants while maintaining satisfactory statistical power.6 For example, a factorial design with two intervention components with two levels each yields four cells (i.e., 2 x 2 = 4), each representing a group of participants assigned to a study condition that receives a unique combination of intervention component levels. As the number of intervention components in the factorial design increases, the number of cells grows exponentially (i.e., four components with two levels each requires 2 x 2 x 2 x2 = 16 cells). Because participants in factorial experiments are independently assigned to a level on each factor and factors are analyzed separately for main effects, statistical power will generally be equivalent to a single-factor RCT that has the same number of study arms as the factorial design’s number of levels within each factor.
Despite the benefits that factorial designs offer with regard to sample size and statistical power, they also present complexity and challenges for subject allocation, especially when the number of cells is large. Consensus guidelines for reporting randomized trials (i.e., Consolidated Standards of Reporting Trials (CONSORT)7) describe a range of acceptable methods for allocation of participants to study cells and suggest that three criteria are important for determining which method to use. First, participant allocation should result in balanced sample sizes across study conditions to maximize statistical power.7–9 Second, participant allocation should result in study conditions that are equivalent with respect to covariates that are expected to impact intervention outcomes (i.e., equivalent groups;10). Third, participant allocation should be completely unpredictable to both study staff and to participants so as to ensure that measured and unmeasured participant characteristics, and selection biases in general, do not influence participants’ assignment to conditions. Given the large number of cells in factorial experiments and division of participants across those cells, balanced sample sizes and equivalent groups are especially important, yet may be difficult to achieve. Finally, we suggest additional criteria that are common to many practical decisions: cost and complexity including resource utilization. Some allocation procedures can be readily implemented using a range of accessible methods and software, whereas other methods may require coding or specialized software that must then be incorporated into workflow. The four outcomes of interest in the present study are these four criteria for subject assignment methods: balance of sample size, equivalence of groups, unpredictability, and complexity.
The CONSORT statement7 classifies the range of acceptable assignment methods into three categories: Simple randomization, which includes the use of random number tables, computerized random number generators, or even a coin toss. Restricted randomization, which involves combining random assignment with additional strategies to improve balance and equivalence across cells. For example, assigning participants in blocks that are the same size as (or a multiple of) the number of cells promotes balanced sample size across conditions.7,9 Stratification defines subsets of participants within which random assignment with blocking occurs,7 thus promoting equivalence in the baseline characteristics used to define the strata.11
Lastly, adaptive randomization procedures show advantages over more traditional restrictive and simple randomization procedures.12 Maximum tolerated imbalance (MTI) represents a class of more novel adaptive randomization procedures which defend against selection bias by implementing simple randomization until a pre-defined imbalance in sample sizes occurs, at which point a “big stick” is used to deterministically regain balanced sample sizes across conditions13. For cases in which equivalence of covariates is of utmost concern rather than balanced sample sizes, covariate adaptive randomization strategies such as minimal sufficient balance (MSB) can be used. MSB uses simple randomization until inequivalence on a covariate is reached, which is determined quantitatively by pre-specified p-value limits from t-tests; when the p-value limit is reached on a covariate across conditions, a more predictable assignment, such as biased coin assignment, is implemented to achieve equivalence once again.14,15 In contrast, Minimization involves assigning each participant to the condition that minimizes differences in sample size and pre-specified covariates across the all of the study cells.7 In other words, participants are allocated to the cell that would result in the minimum sum of ranges of both sample size and covariates if the participant were to be assigned to each possible cell. Although this appears fundamentally deterministic, a random element can be introduced to settle ties among study cells.16 In the case of a factorial design with 16 cells, sample size and covariate ties become more common occurrences; randomized allocation between tied cells becomes a logical and necessary technique to incorporate into the minimization procedure.
As expected, each of these subject allocation methods theoretically has strengths and weaknesses with regard to balance, equivalence, unpredictability, and complexity (see Table 1). With regard to predictability, simple randomization is the most unpredictable on a theoretical level and is therefore best for reducing the threats of selection bias.9,12 In comparison, restricted randomization includes a variety of procedures, each with varying levels of selection bias threat. Blocking heightens selection bias because block size is generally known to study investigators, and assignment becomes increasingly predictable as cumulative enrollment reaches numbers equivalent to multiples of the number of study conditions.7,17 While introducing random variation in block size (i.e., permuted blocks) can mitigate this problem,7 the benefit of doing so with respect to balance declines as the number of study cells increases—as is often the case for factorial experiments. In contrast, adaptive MTI and MSB procedures minimize selection bias with the use of simple randomization up until implementation of a more deterministic method becomes necessary based on pre-determined limits.13,15 Thus, the degree of predictability of restricted randomization and adaptive randomization depends on the exact procedure used and the number of arms or factors in the study design to which it is applied.
Conversely, minimization without a random element is an inherently deterministic procedure that can be predicted given perfect knowledge of prior assignments and covariate data for the next participant being assigned, as well as the algorithm by which these values result in assignment. The more covariates included in the minimization algorithm, the more difficult it would theoretically be for an investigator to keep track of such information mentally; nevertheless, selection bias remains a pitfall of this purely deterministic method, and even simple guessing rules may have the potential to exceed chance levels.18 Adding elements of random assignment into minimization algorithms is preferable because such methods reduce reliance on deterministic allocations, and reduce the likelihood of an investigator’s guess of an assignment being correct.7 Thus, for minimization, predictability again depends on the exact procedure used and the study design to which it is applied.
Table 1. Theoretical comparison of allocation methods
|
Unpredictability
|
Balanced sample sizes across conditions
|
Equivalent baseline characteristics across conditions
|
Cost & complexity
|
Simple randomization
|
Best; random assignment prevents predictability
|
Poor; likely to result in differences across cells
|
Poor; likely to result in differences across cells
|
Best; simple to implement
|
Stratification with permuted blocks
|
OK; Random order of assignments within blocks within strata reduces predictability but known block sizes increase predictability
|
Very good; blocking improves balance, but this is mitigated by stratification
|
Good; stratification improves equivalence on specific variables
|
Very good; more complex, but solutions are widely available
|
Maximum tolerated imbalance
|
Very good; random assignment protects against selection bias until big stick is needed
|
Very good; results at or below maximum tolerated imbalance of samples
|
Poor; No better than simple randomization
|
OK; can be implemented in a range of available software, but requires coding
|
Minimal sufficient balance
|
Very good; random assignment protects against selection bias until biased coin is needed
|
Poor; No better than simple randomization
|
Very good; results at or below maximum tolerated inequivalence of covariates
|
OK; can be implemented in a range of available software, but requires coding
|
Minimization
|
Poor when purely deterministic; improved with incorporation of random element
|
Very good; should promote balance, depending on algorithm
|
Best; promotes equivalence on a large number of variables
|
OK; can be implemented in a range of available software, but requires coding
|
When seeking balanced sample sizes and groups that are equivalent with respect to baseline variables across study conditions, simple randomization is expected to perform the most poorly.9 The credibility of factorial experiments can be significantly compromised by simple randomization because of the compounded problem of yielding cells that are imbalanced with respect to sample size and non-equivalent with respect to key covariates.7 Many researchers continue to follow the precedent of using stratification with permuted blocks to address these issues; however, making such determinations based on precedent can be misguided and ignores the substantial threat of selection bias that blocking creates.19,20 An additional limitation to stratification with permuted blocking is that when block size is equivalent to the number of study conditions, stratification is only feasible for two or three variables at most.11 In the context of factorial designs, stratification with permuted blocks is therefore additionally limited in the number of variables on which it can promote equivalence. MTI and MSB methods protect from selection bias with a default to simple randomization, but only up until the limit on tolerated imbalance or inequivalence is reached. MTI satisfies the need for a pre-determined level of balance on sample size, whereas MSB supports equivalence on selected covariates. In factorial design studies requiring both balance and equivalence, implementing one of these techniques alone will not be sufficient.
Minimization procedures can ensure that conditions have balanced sample sizes and equivalent baseline characteristics for a large number of variables, even for studies with small sample sizes and/or many treatment conditions, across all stages of an experiment. Some therefore argue that minimization procedures are not only acceptable, but a superior alternative to simple or restricted randomization techniques such as stratification with permuted blocks.21 In minimization assignment, the first patient is assigned at random, and each following participant is assigned to the condition that minimizes differences across study conditions with respect to sample size and selected covariates. Assignment becomes less easily guessed correctly by researchers as more variables are added and the minimization algorithm becomes more complex.22 Moreover, researchers may incorporate randomization into their minimization scheme. For study designs with more than two treatment conditions, randomization may also be necessary if the minimization algorithm results in ties among two or more conditions. Researchers may even choose to set up the minimization algorithm to incorporate randomization for near-ties or to use a weighted probability that favors, but does not determine, assignment to the condition that minimizes imbalances.7 Such methods appear to be effective mechanisms to reduce the risk of minimization assignment from being fully "deterministic".16
Finally, assignment strategies differ with respect to cost and complexity. In this regard, simple randomization is arguably best as a wide range of software and even a coin or die can be sufficient. Stratification with permuted blocks is a close second because it is embedded in a range of software packages used by researchers, including RedCap. While simple conceptually, MTI, MSB, and minimization procedures are currently the most difficult to implement, requiring specialized coding in software packages like Excel, Stata, or R. These randomization algorithms must be individualized based on covariates of interest, thresholds of acceptable imbalance or inequivalence, and number of arms in particular studies. Once the minimization and MSB programs are written, study staff must obtain sufficient covariate data (e.g., age, ethnic identity, co-morbid conditions) before randomizing participants to run the program. MTI, MSB, and minimization programs will likely be stand-alone as none of these procedures are currently integrated into other commonly used research study management systems such as RedCap or StudyTrax. Overall, MTI, MSB, and minimization are the most complex because they require additional skills, staff time, and resources.
Aims of the Current Study
Given the relative strengths and weakness of simple randomization, stratification with permuted blocks, MTI, MSB, and minimization procedures for participant assignment with respect to a given study design (e.g., number of cells, sample size) and hypotheses (e.g., number of conditions of interest, importance of testing interactions), it is critical that researchers make deliberate, informed choices about their participant assignment procedures for their unique studies, particularly in the context factorial designs within MOST frameworks. The primary aim of this paper is to present a case study of how assignment procedures can be directly compared by conducting simulations drawing from a prior locally-collected dataset.