Keyboard: An R Package Suite for Early Phase Dose-finding Designs


 Background: Phase I and/or I/II oncology trials are conducted to find the maximum tolerated dose (MTD) and/or optimal biological dose (OBD) of a new drug or
treatment. In these trials, for cytotoxic agents, the primary aim of the single-agent or drug-combination is to find the MTD with a certain target toxicity rate, while for the cytostatic agents, a more appropriate target is the OBD, which is often defined by consideration of toxicity and efficacy simultaneously. However, there still lacks accessible software packages to achieve both yet.
Results: Objective of this work is to develop a software package that can provide tools for both MTD- and OBD-finding trials, which implements the Keyboard design for single-agent MTD-finding trials by Yan et al., the Keyboard design for drug-combination MTD-finding trials by Pan et al., and phase I/II OBD-finding method by Li et al., in a single R package, called Keyboard. For each of the designs, the Keyboard package provides corresponding functions that begins with get.boundary( . . . ) to determine the optimal dose escalation and de-escalation boundaries, that begins with select.mtd( . . . ) to select the MTD when the trial is completed, that begins with select.obd( . . . ) to select the OBD at the end of a trial, and that begins with get.oc( . . . ) to generate the operating characteristics.
Conclusions: The developed Keyboard R package provides convenient tools for designing, conducting and analyzing single-agent, drug-combination and phase I/II dose-finding trials, which supports Bayesian designs of innovative dose-finding studies.


Introduction
A phase I clinical trial is critical in new drug/treatment development because it determines the dose that will be further investigated in the subsequent phase II or III trials. For the cytotoxic agent, one primary objective of a phase I dose-finding trial is to find the maximum tolerated dose (MTD), which is defined as the highest dose that has a dose-limiting toxicity (DLT) rate less than or close to a prespecified target rate. The identified MTD will then be employed in later phases, for example, phase II clinical trials. Statistical methods for the single-agent MTD-finding designs include the algorithm-based designs, such as the 3 + 3 design [1], the biased-coin design [2,3,4]; the model-based designs, such as the continual reassessment method (CRM) [5] and related methods [6], and the model-assisted design designs, such as mTPI [7], BOIN [8], Keyboard [9] designs, etc.
It should be noted that all the above methods were developed for the cytotoxic agent to find the MTD. The cytotoxic drug development, however, is rested on the premise that agents must be "cytotoxic" to be effective. By equating efficacy with the toxicity, traditional drug development, progressing from phase I through phase III clinical studies, has sought the highest effective dose that does not induce intolerable levels of toxicity, which essentially justifies the goal of finding the MTD for the phase I dose-finding trials. This assumption may be reasonable for cytotoxic agents, however, may not hold for the cytostatic or molecularly targeted agents. For example, the chimeric antigen receptor (CAR) T therapies require a balance of a boosting of the immune system to combat cancer while avoiding over-stimulation. In this case, preliminary dose exploration should aim to capture effective biologic activity rather than dose-limiting toxicity alone. Therefore, the optimal biological dose (a.k.a, OBD), which is defined as the lowest dose with the highest rate of efficacy while safe, is a more appropriate endpoint than the MTD since it takes accounts of both the toxicity and efficacy without the above toxicity-and-efficacy-simultaneously-increase-with-dose assumption. In literature, the associated method is termed as the phase I/II OBD-finding design. There are many proposed methods for the OBD-finding trials, for example the model-based methods [10,11,12,13,14] and model-assisted methods [15,16,17,18,19,20] Among these designs, Li et al. [15] proposed a toxicity and efficacy probability interval (TEPI) design, which was shown to have desirable operating characteristics and is simple and transparent to clinicians with a physician-elicited decision table that maps the two-dimensional probability intervals to a set of dosing decisions. The Keyboard package develops multiple R functions to implement this design since the method behind this design is also interval-based which is the core for the Keyboard design.
Except the single-agent trials, nowadays treating patients with a combination of agents is prevalent in clinical trials, especially for the oncology treatments as they are more effective and less susceptible to drug resistance than are single-agent trials. Trial designs for drugcombination studies involve several distinct features that are beyond the scope of methods for single-agent studies. A major challenge in designing combination trials is that dose combinations are only partially ordered in terms of toxicity probability. For instance, consider a trial combining m doses of agent A and n doses of agentthat is, we now have a n × m dose matrix, a major challenge in designing combination trials is that dose combinations are only partially ordered in terms of toxicity probability, that is, a priori, we cannot fully rank n × m dose matrix from low to high by their toxicity probabilities. Various designs have been proposed for the drug-combination MTD-finding trials: a design based on the order of the restricted inference [4], a copula-type regression model [19], latent contingency tables [20], the partial order CRM method (POCRM) [21], sequential dose-finding strategy [22,23], a Bayesian optimal interval design [24], Keyboard combination design [27], and Bayesian data augmentation for late-onset toxicity [28], and among others. The Keyboard package implements the Keyboard combination design proposed by Pan et al. [27].
In sum, we develop an R package, Keyboard, to include three methods in [9,15,27] for practitioners' easy access to these methods. Since all of these methods use the Bayesian interval-based method, which is the underlying theoretical basis for the Keyboard design, therefore, we collect these three methods in one package.
The paper is organized as follows. We concisely introduce the three designs, Keyboard single-agent method for the MTD [9], Keyboard combination design [27], and Keyboard phase I/II OBD design [15] in Section 2. Section 3 describes the functions for implementing these methods. Section 4 provides three exemplary trials and demonstrates how to use the Keyboard package to design trials for single agent, drug combination of MTD-or OBD-dose finding.
Section 5 ends in a conclusion.

Design for the MTD dose-finding single-agent trials
The design in this package for the single-agent MTD-finding was proposed by Yan et al. [9], and its theoretical properties were further explored by Pan et al. [27]. We call this design single agent Keyboard design, or simply, the Keyboard design. This design uses the toxicity probability's posterior distribution to guide dose transition. To decide whether to escalate or de-escalate the dose, we need to first find the strongest key, which is the interval with the maximum posterior probability. If this key is to the left of the "target key", then we escalate the dose because the data suggest that the current dose is most likely too low; if this key is to the right of the target key, then we de-escalate the dose because the observed data suggest that the current dose is likely too toxic; and if the strongest key is the target key, then we stay at the current dose because the observed data support the notion that the current dose is most likely to be in the right dose interval (See Figure 1).
Let φ be the target toxicity rate specified by the investigator and p d ∈ (0, 1) denote toxicity probability of dose level d ∈ {1, · · · , D}. At any time point, assume n d patients have been treated at the current dose d, and y d of them experienced dose-limiting toxicity (DLT). Assuming that D d = (n d , y d ) given the observed data, a beta-binomial model is used: . By specifying a target toxicity interval (is also called toxicity equivalent interval in the mTPI design) I target = (φ − 1 , φ + 2 ) , which is named as a target key in the Keyboard design, here 1 and 2 are small positive values, for instance, both are 0.05. Then, using the width of the target key divides [0,1] to a series of equally-wide keys/intervals, denoted by I k .
Dose-transition rules of the Keyboard design are as follows: • To decide whether to escalate or de-escalate the dose, the Keyboard design calculates the strongest key I max , which is defined as:

Keyboard design for drug-combination MTD-finding trials
Drug-combination trials are challenging due to possible partial orderings among the dose combination (see Figure 2) and the larger search space (e.g., the dimension of the dose space expands in a multiplicative manner). Though there are several proposed combination designs in literature [12,20,27] and their associated statistical performances demonstrated superior theoretically and empirically, their usages are still limited in practice possibly due to either statistical or computational complexity. The Keyboard combination design proposed by Pan et al. in [27] circumvents the challenges by extending the above-introduced single-agent Keyboard design to the dual-agent setting.
Let p jk be toxicity probability of the dual agents agent A at level j and agent B at level k, n jk is the number of subjects treated at the dose combination (j, k) and y jk is the number of subjects who experienced DLT at the dose combination (j, k).
Algorithm of the Keyboard combination design is as below: Step 1. Treat the first cohort of subjects at the dose combination (1, 1).
Step 2. At the dose combination (j, k), given the observed data D jk = (n jk , y jk ), find the strongest key I max based on the posterior distribution of p jk : If I max ≺ I target , then we escalate the dose; if I max I target , then we deescalate the dose; otherwise, if I max ≡ I target , then we stay at the current dose.
Step 3. The process continues until the pre-specified maximum sample size N is achieved.
Here, I 1 ≺ ( )I 2 means that I 1 is at the left (right) of the I 2 .
However, for combination trials with dual-agents, the difficulty is that when we decide to escalate/de-escalate a dose, there usually exist more than one option, for example, if the decision is to escalate, we may move either dose level of agent A, dose level of agent B, or both simultaneously, to it (their) adjacent higher dose level(s).
In [27], the authors defined the admissible dose escalation/de-escalation sets as: For instance, A E 1 means that escalation can be allowed to move the agent A from j to j + 1, or move the agent B from k to k + 1. We can see that the diagonal movement is prohibited in escalation/de-escalation due to safety concerns.

Keyboard: An R Package Suite for Early-phase Dose-finding Designs
The dose assignment algorithms is then as follows: • Escalation: escalate to the dose combination that belongs to A E 1 and has the highest value of Pr(p j k ∈ I target |D jk ) where (j , k ) ∈ A E 1 .
• De-escalation: de-escalate to the dose combination that belongs to A D 1 and has the highest value of Pr(p j k ∈ I target |D jk ) where (j , k ) ∈ A D 1 .
If there are multiple optimal dose combinations with the same value of Pr(p j k ∈ I target |D jk ), then we randomly choose one. The trial is completed when the maximum sample size is reached. Given all observed data at the completion of a trial, we use the matrix isotonic regression to obtain the estimates of p jk 's value and select the MTD as the combination with a estimated toxicity probability that is closest to the target. Similar safety rules to the single-agent Keyboard design are also used here.

Design for the single-agent OBD-finding trials
Traditionally, the purpose of a dose-finding design in cancer is to find the maximum tolerated dose (MTD) based solely on the toxicity. However, for the molecular targeted agents, little toxicity may arise within the therapeutic dose range and the dose-response curves may not be strictly monotonic. This challenges the conventional principle of "more is better". Instead, the optimal biological dose (OBD), which is defined as the lowest dose with the highest rate of efficacy while safe, is a more appropriate endpoint. A study by Corbaux et al. [29] showed that the dose approved by the FDA is consistent with the OBD for 83% of the drugs in a total of 87 completed trials for evaluating molecular targeted agents.
The design for finding the OBD can also base on the posterior distributions of the toxicity and efficacy rates jointly to guide the dose-transition. Similar to the above Keyboard paradigm, a highest joint unit probability mass (JUPM) which is defined by the joint probabilities falling into the pre-specified equivalence target intervals for the toxicity and efficacy targets (the formal definition will be introduced shortly) and is also referred to as the winner key, is used as the cornerstone for making dose-assignment decisions for the OBD-finding studies. The Keyboard package implements a method proposed by Li et al. in [15].
To use this design, a boundary table for both the toxicity and efficacy needs to be prespecified by the investigators prior to the study. To be specific, for the toxicity, taking a four-subinterval schema as an example, it can be categorized as: low, moderate, high, and unacceptable. Generating these four intervals requires the following three input parameters, toxicity.low, toxicity.moderate, toxicity.high, to form the sub-intervals of (0, toxicity.low), (toxicity.low, toxicity.moderate), (toxicity.moderate, toxicity.high), and (toxicity.high, 1). These parameters are specified prior to a study by investigators and an example will be shown shortly.
Similarly, a four-subinterval for the efficacy can be generated if three parameters, efficacy.low, efficacy.moderate, efficacy.high, are given.
Assume there are d doses in the trial and the current dose is i, number of patients at this dose level is n i , the number of patients who experienced toxicity is x i , and the number of responses is y i . The trial data can be represented as follows: Assuming that the toxicity probability is p i and the efficacy probability is q i at dose level i, the probability unit intervals for toxicity and efficacy can be partitioned into subintervals (a, b) and (c, d). Here, (a, b) is the subinterval for the toxicity probability, and (c, d) is the subinterval for the efficacy probability. The (a, b) × (c, d) represents a combination interval for joint toxicity and efficacy probability.
For instance, given the above four-subinterval for toxicity and efficacy probability, there are 16 combination intervals in total. Investigators would then be required to provide associated 16 decisions corresponding to the combination intervals based on the clinical rationale, which is quite similar to pre-specify utility scores strategy adopting by various authors [19,20].
Decision "D, E, and S" denotes de-escalation, escalation, and stay. An exemplary pre-specified boundary table with a target toxicity rate of 0.2 and a target efficacy rate of 0.4 is given in Table 2. For making the decision, we need to calculate the joint unit probability mass (JUPM) and the JUPM is defined as follows: Here, P r(p j ∈ (a, b), q j ∈ (c, d)|D) is the posterior probability of p i and q i falling in the subinterval (a,b) and (c,d). Assume the priors for both p i and q i follow independent beta distributions Beta(α p , β p ) and Beta(α q , β q ) independently. The posterior distributions for p i and q i are Beta(α p + x i , β p + n i − x i ) and Beta(α q + y i , β q + n i − y i ). Using these posterior distributions to update the JUPMs for all sixteen combination intervals, we can find the winning combination interval (a*, b*) and (c*, d*), which has the largest JUPM value, which can guide the dose-transition to treat the next cohort of patients.
Two dose exclusion rules in terms of safety and futility are recommended in real practices similar to designs introduced in previous sections: Safety rule: Similar to the previous methods, if at least 3 patients have been treated at a given dose and the observed data indicate that the probability of the current dose's toxicity rate exceeding the target toxicity rate by more than 95%, then we eliminate the current dose and any higher doses from the trial to avoid exposing future patients to unacceptably toxic doses. If the lowest dose is unacceptably toxic, then the trial terminates early and no dose is selected as the OBD. This corresponds to a dose assignment of "DUT" output by the software, which means to de-escalate because of unacceptable high toxicity and exclude the current dose and any dose higher than this dose from the trial.
The probability threshold can be specified with cutoff.eli.toxicity in the software shown in the next section.
Futility rule: if at least 3 patients have been treated at a given dose and the observed data indicate that the probability of the current dose's efficacy rate exceeding the target efficacy rate by less than 30%, then we eliminate this dose from the trial to avoid exposing future patients to these futile doses. The probability threshold can be specified with cutoff.eli.efficacy. The software will output two possible dose assignments: "EUE" and "DUE", both of which exclude the current dose from the trial. "EUE" denotes escalation because of unacceptable low efficacy; "DUE" denotes de-escalation because of unacceptable low efficacy.
An attractive feature of this design is that its dose escalation and de-escalation rule can also be tabulated prior to the study. Thus, when implementing a trial, no real-time calculation or complicated model fitting is needed, and we need only to count the number of patients, the number of DLTs, and the number of responses observed at the current dose, and the decision to escalate or des-escalate the dose is based on the pre-tabulated decision rules shown in Table   2.
For the final OBD selection after completing a trial, a utility score is evaluated to quantify the desirability of all admissible doses. Calculation of utility scores requires the posterior probabilities for toxicity p i and efficacy q i , which can be computed using Beta(α p + x i , β p + n i − x i ) and Beta(α q + y i , β q + n i − y i ), assuming that the priors for both p i and q i follow independent beta distributions Beta(α p , β p ) and Beta(α q , β q ). In this package, we provide the three options for the utility functions.
The first utility function is a function of the toxicity f 1 (p), where p denotes the toxicity Keyboard: An R Package Suite for Early-phase Dose-finding Designs rate, and the efficacy f 2 (q), where q denotes the efficacy rate. f 1 (p) is shown as in equation (1).
Here, p 1 is the cutoff lower limit and p 2 is the cutoff upper limit for safety utility function is f 1 (p), which was used in [15].
Here, q 1 is the cutoff lower limit and q 2 is the cutoff upper limit for safety utility function is f 2 (q).
Assuming the toxicity and efficacy are independent of each other, the first utility function that quantifies benefit-risk trade-off at current dose i is defined as follows: The second utility function depends on a marginal toxicity probability π T,i = P r(Y T = 1|d = i) and a marginal efficacy probability π E,i = P r(Y E = 1|d = i), which is defined as follows: where, w 1 is a pre-specified weight. This trade-off function describes how much patients are willing to trade an increase of w 1 in the DLT rate for a unit increase in the efficacy rate. If w 1 =0, we obtain a special case that the dose with the highest efficacy is the most desirable.
The third utility function also depends on the marginal toxicity probability π T,i = P r(Y T = 1|d = i) and efficacy probability π E,i = P r(Y E = 1|d = i), but it puts an additional penalty for over-toxicity and is defined as follows: where w 1 and w 2 are pre-specified weights, I(·) is an indicator function, and ρ is a prespecified toxicity threshold deemed of substantial concern and can be choosen as the target toxicity rate. Compared to the above second utility function (4), this trade-off function is more flexible and allows to impose a higher penalty (i.e., w 1 + w 2 ) when the true DLT rate π T,j exceeds the threshold ρ.
Once the utility score is computed for all the doses, then the optimal biological dose (OBD) can be estimated by:

Results
The R package Keyboard contains functions that implement the designs introduced in the above section for the single-agent, drug-combination MTD-finding trials and the phase I/II OBD-finding trials.

Single-agent MTD-finding design
• get.boundary.kb(· · · ): This function is used to generate the optimal dose-escalation and de-escalation boundaries for conducting a single-agent trial with the Keyboard design.
• select.mtd.kb(· · · ): This function is used to select the MTD after the single-agent trial is completed.
• next.comb.kb(· · · ): This function is used to determine the dose combination for the next cohort of patients in drug-combination trials that aim to find one MTD.
• select.mtd.comb.kb(· · · ): This function is used to select the MTD after the drugcombination trial is completed.
• get.oc.comb.kb(· · · ): This function is used to generate the operating characteristics of the Keyboard design for drug-combination trials.
Phase I/II trial OBD-finding design • get.decision.obd.kb(· · · ): This function is used to generate the boundary table and decision matrix for single-agent phase I/II trials designed to find the OBD.
• select.obd.kb(· · · ): This function is used to select the OBD at the end of a trial.
Phase I/II trial-automatically generating decision tables Different from the above, the following functions do not rely on pre-specified decision tables (Details and example are introduced shortly).
• get.decision.obd2.kb(· · · ): This function is used to provid the boundary table and decision matrix for the phase I/II trials automatically.
• select.obd.kb(· · · ): This function is used to select the optimal biological dose (OBD) at the end of a single-agent phase I/II trial.
• get.oc.obd2.kb(· · · ): This function is used to generate the operating characteristics, and the decision tables is generated automatically.

Single-agent trials
Design and conduct the trial To design a single-agent trial, we can run the function get.boundary.kb(·) to obtain the dose escalation and de-escalation boundaries. This function has the following arguments: • target The target dose-limiting toxicity (DLT) rate.
• ncohort The total number of cohorts.
• cohortsize The number of patients in the cohort.

• marginL
The difference between the target and the lower bound of the "target key" (proper dosing interval) to be defined. The default is 0.05.

• marginR
The difference between the target and the upper bound of the "target key" (proper dosing interval) to be defined. The default is 0.05.
• n.earlystop The early stopping parameter. If the number of patients treated at the current dose reaches n.earlystop, then stop the trial and select the MTD based on the observed data. The default value is 100.
• cutoff.eli The cutoff value to eliminate an overly toxic dose and all higher doses for safety. The recommended value is 0.95.
• extrasafe Set extrasafe = TRUE to impose a stricter stopping rule for extra safety, expressed as the stopping boundary value in the result.
As introduced previously, there are two built-in stopping rules: (1) Stop the trial if the lowest dose is eliminated due to unacceptably high toxicity. In this case, no dose should be selected as the MTD/OBD; (2) Stop the trial and select the MTD if the number of patients treated at the current dose reaches n.earlystop.
The first stopping rule is a safety rule to protect patients from the case in which all doses are overly toxic. The rationale for the second stopping rule is that when the number of patients assigned to a dose is large (i.e., n.earlystop), the dose-finding algorithm has approximately converged, or due to limited sample size of rare diseases, we can sometimes stop the trial if a certain number of patients have been treated at a dose level so that we can stop the trial early and select the MTD to save sample size and reduce the trial's duration. The trade-off is that it may affect the MTD selection percentage and decrease the rate of stopping for safety if the first dose is overly toxic. The value of n.earlystop should be calibrated by simulation to obtain desirable operating characteristics. In general, we recommend n.earlystop = 9 or 12 often has desirable operating characteristics. Our experience is that this stopping rule is particularly useful when there is strong prior knowledge that the first dose is safe because a major side effect of using the stopping rule is that it decreases the rate of stopping for safety when the first dose is actually overly toxic.
Although the Keyboard design has a built-in safety stopping rule (i.e., stopping rule (1) described above), for some applications, investigators may prefer a stricter stopping rule for extra safety when the lowest dose is possibly overly toxic. Setting extrasafe = TRUE imposes the following stronger stopping rule: Stop the trial if (1) the number of patients treated at the lowest dose ≥ 3, and (2) Pr(toxicity rate of the lowest dose > target | data) > cutoff.eli -offset.
Note that as a trade-off, the stricter stopping rule will decrease the MTD selection percentage when the lowest dose actually is the true MTD. When using the option extrasafe = TRUE, we recommend the default value offset = 0.05, but users can calibrate the value of offset to obtain desired operating characteristics. In practice, offset is rarely greater than 0.2.
As an example, suppose we want to conduct a phase I trial with J = 5 dose levels and a target toxicity rate of φ = 0.3. The maximum sample size is 30 patients, and patients are treated in cohorts of size 3. Using the default values of marginL, marginR, and cutoff.eli au-tomatically provided by the function, we can design the trial by running get.boundary.kb(·): R> get.boundary.kb(target=0.3, ncohort=10, cohortsize=3) The following are decision boundaries: Number of patients treated 3 6 9 12 15 18 21 24 27 30 Escalate if # of DLT <= 0 1 2 2 3 4 5 5 6 7 De-escalate if # of DLT >= 2 3 4 5 6 7 8 9 10 11 Eliminate if # of DLT >= 3 4 5 7 8 9 10 11 12 14 A more complete version of the decision boundaries is given as follows: Number of patients treated 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17   18 19 20 21 22 23 24 25 26 27 28 29  Eliminate if # of DLT >= NA 2 3 3 4 4 5 5 5 6 6 7 7 8 8 8 9   9 9 10 10 11 11 11 12 12 12 13 13 14 Remarks: • The output presents the dose escalation/de-escalation rules in two forms: based on the observed toxicity rate, and based on the observed DLT. We recommend the latter because clinical researchers often find it easier to use. Obtain the operating characteristics For the protocol preparation, it is often useful to obtain the operating characteristics of the design. The function get.oc.kb(·) can be used for this purpose. This function shares the same set of arguments as the function get.boundary.kb(·) described previously, with three additional arguments: • p.true A vector containing the true toxicity probabilities of the investigational dose levels.

• startdose
The starting dose level for treating the first cohort of patients. The default value is startdose = 1, i.e., starting from the lowest dose.
• ntrial The total number of trials to be simulated. The default value is 1000.
Using the same setting as above and assuming that the true toxicity scenario is p.

Select the MTD when the trial is completed
When the trial is completed, we can select the MTD based on the observed data using the function select.mtd.kb(· · · ). This function has six arguments: target, npts, ntox, cutoff.eli, extrasafe and offset, where • npts A vector containing the number of patients treated at each dose level.  We note here that the above functionals can also be implemented by Keyboard design module via Shiny app at www.trialdesign.org.

Design and conduct the trial
To design a drug-combination trial, we can run the function get.boundary.comb.kb(·) to obtain the dose escalation and de-escalation boundaries, which are used to run the trial.
This function has the following arguments: • target The target DLT rate.
• ncohort The total number of cohorts.
• cohortsize The number of patients in the cohort.

• marginL
The difference between the target and the lower boundary of the "target key" (proper dosing interval) to be defined. The default is 0.05.

• marginR
The difference between the target and the upper boundary of the "target key" (proper dosing interval) to be defined. The default is 0.05. • cutoff.eli The cutoff value to eliminate an overly toxic dose and all higher doses for safety. The recommended value is 0.95.
• extrasafe Set extrasafe = TRUE to impose a stricter stopping rule for extra safety, expressed as the stopping boundary value in the result.
As an example, suppose we want to conduct a phase I trial with a target toxicity rate of φ = 0.3. The maximum sample size is 30 patients, and patients are treated in cohorts of size 3. Using the default values of marginL, marginR, and cutoff.eli automatically provided by the function, we can design the trial by running get.boundary.kb(·): R> get.boundary.comb.kb(target=0.3, ncohort=10, cohortsize=3) The following are decision boundaries: Number of patients treated 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17   18 19 20 21 22 23 24 25 26 27 28 29 30 Escalate if # of DLT <= 0 0 0 0 1 1 1 1 2 2 2 2  Obtain the operating characteristics The function get.oc.comb.kb(·) can be used to obtain the operating characteristics of the Keyboard drug-combination design. This function shares the same set of arguments as the function get.oc.kb(·), except that p.true is now a matrix (rather than a vector) and startdose is a vector of length 2 (rather than a scalar).

Conduct the trial
The function next.comb.kb(·) is used to conduct phase I drug-combination trials that aim to find a single MTD. It takes the data from patients who have been enrolled into the trial as the input, and outputs the dose combination for treating the next cohort of new patients. The function next.comb.kb(·) shares a similar set of arguments with the function get.boundary.kb(·) described previously, with three additional arguments: • npts A matrix recording the number of patients treated at each dose combination.
• ntox A matrix recording the number of patients who experienced toxicity at each dose combination.
• dose.curr The current dose combination (the dose combination that was used to treat the most recently enrolled cohort of patients).
Suppose that we conduct a 3 × 4 drug-combination trial with 3 dose levels of agent A and 4 dose levels of agent B, aiming to find a MTD that has a target toxicity rate of 0.3.
The maximum sample size is 48 patients, and patients are treated in cohort sizes of 3. Let To determine the dose for the third cohort of patients, we again call next.comb.kb(·) with updated y, n and dose.curr, as follows: The recommended dose combination for the next cohort of patients is (2, 2) Therefore, we should de-escalate the dose and treat the third cohort of patients at dose (2,2). We repeat this procedure until the maximum sample size is reached.

Select a MTD when the trial is completed
When the trial is completed, based on the observed data, we can select a MTD using the function select.mtd.comb.kb(·). This function has seven arguments: target, npts, ntox, The result is that dose combination (2, 2) is selected as the MTD. This combination design can also be implemented at www.trialdesign.org.

Design and conduct the trial
To design a phase I/II trial for finding the OBD, we need to run the function get.decision.obd.kb(·) to obtain the boundary table and decision table. This function has the following arguments to input: • toxicity.low The upper boundary for the low toxicity interval.
• toxicity.moderate The upper boundary for the moderate toxicity interval.
• toxicity.high The upper boundary for the high toxicity interval.
• efficacy.low The upper boundary for the low efficacy interval.
• efficacy.moderate The upper boundary for the moderate efficacy interval.
• efficacy.high The upper boundary for the high efficacy interval.
• target.toxicity The target DLT rate.
• target.efficacy The target efficacy rate.
• ncohort The total number of cohorts.
• cohortsize The number of patients in the cohort.
• cutoff.eli.toxicity The cutoff value to eliminate a dose with an unacceptably high toxicity for safety. The default value is 0.95.
• cutoff.eli.efficacy The cutoff value to eliminate a dose with unacceptably low efficacy. The default value is 0.3.
As an example, suppose we want to conduct a phase I/II trial with J = 5 dose levels, a target toxicity rate of target.toxicity = 0.20, a target efficacy rate of target.ef f icacy = 0.40.
Patients are treated in cohorts of size 3 and ncohort is 10. • D: deescalate to the previous dose level j-1, assuming that the current dose level is j and j is not the lowest dose level. Otherwise, treat the next cohort of patients at the current dose level j.
• E: escalate to the next dose level j+1 assuming that the curent dose level is j and j is not the highest dose level. Otherwise, treat the next cohort of patients at the current dose level j.
• S: stay at the current dose level j (treat the next cohort of patients at dose level j).
• EUE: escalate to the next higher dose level because of unacceptably low efficacy.
• DUE: deescalate to the previous lower admissible dose level because of unacceptably low efficacy.
• DUT: deescalate to the previous lower admissible dose level because of unacceptably high toxicity.

Keyboard: An R Package Suite for Early-phase Dose-finding Designs
• Default stopping rule: if P r(p i > p T |D) is too high ( the default is 0.95), then any dose higher than i or equal to i is excluded from the trial.
• Default futility rule: if P r(q i > q E |D) is too low( the default value is 0.3), then dose i is excluded from the trial.
From the above output, we can firstly have a pre-specified decision table like the one shown by Table 2 in the previous section and then get the decision rules for all combinations of toxicity and response. For example, we can see that if there are 3 patients on a specific dose level with 1 toxicity and 1 response, the decision would be "S", that is, we will continue to administer the current dose to the next cohort of patients.

Obtain the operating characteristics
The function get.oc.obd.kb(·) can be used to obtain the operating characteristics of the design. This function takes these following arguments: • toxicity.low The upper boundary for the low toxicity interval.
• toxicity.moderate The upper boundary for the moderate toxicity interval.
• toxicity.high The upper boundary for the high toxicity interval.
• efficacy.low The upper boundary for the low efficacy interval.
• efficacy.moderate The upper boundary for the moderate efficacy interval.
• efficacy.high The upper boundary for the high efficacy interval.
• target.toxicity The target DLT rate.
• target.efficacy The target efficacy rate.
• ncohort The total number of cohorts.
• cohortsize The number of patients in the cohort.
• n.early The early stopping parameter. If the number of patients treated at the current dose reaches n.early, then we stop the trial and select the MTD based on the observed data. The default value is 100.
• startdose The starting dose level.
• p.true A vector containing the true toxicity probabilities of the investigational dose levels.
• q.true A vector containing the true efficacy probabilities of the investigational dose levels.
• ntrial The total number of trials to be simulated.
• seed The random seed for simulation.
• p1 The cutoff lower limit for safety utility funciton (3): is the function for toxicity probability, and f(q) is the function for efficacy probability.
• p2 The cutoff upper limit for safety utility function (3).
• cutoff.eli.toxicity The cutoff value to eliminate a dose with unacceptably high toxicity for safety. The default value is 0.95.
• cutoff.eli.efficacy The cutoff value for the futility rule, the acceptably lowe efficacy. The default value is 0.30.
• w1.toxicity The weight for toxicity utility function (4) and (5) The recommended ρ is the target toxicity rate.

Keyboard: An R Package Suite for Early-phase Dose-finding Designs
Using the same setting as above and assuming that the true toxicity scenario is p.true

Select the OBD when the trial is completed
When the trial is completed, we can select the OBD based on the observed data using the function select.obd.kb(· · · ). This function has these arguments: • target.toxicity The target DLT rate.
• target.efficacy The target efficacy rate.

• npts
The vector containing the total number of patients treated at each dose level.
• ntox The vector containing the number of subjects at each dose level who experienced toxicities.
• neff The vector containing the number of subjects at each dose level who experienced efficacies.
• p1 The cutoff lower limit for safety utility funciton (3): U = f (p) × f (q). f(p) is the function for toxicity probability, and f(q) is the function for efficacy probability.
Keyboard: An R Package Suite for Early-phase Dose-finding Designs • p2 The cutoff upper limit for safety utility function (3).
• cutoff.eli.toxicity The cutoff value to eliminate a dose with unacceptable high toxicity for safety. The default value is 0.95.
• cutoff.eli.efficacy The cutoff value for the futility rule, the acceptable lowest efficacy. The default value is 0.30.
• w1.toxicity The weight for toxicity utility function (4) and (5) The recommended ρ is the target toxicity rate.
Assume that the number of patients treated at four doses is npts = (6, 6, 12, 3), the corresponding number of patients who experienced toxicity is ntox = (0, 1, 2, 2), and the corresponding number of patients who experienced efficacy is neff=(4,3,6,1) The OBD using the 1st utility function is dose level 3.
The OBD using the 2nd utility function is dose level 3.
The OBD using the 3rd utility function is dose level 3.

Option 2: Phase I/II trials aiming to find OBD
We also provide a function, without requiring the investigator's efforts to prespecify multiple input parameters/arguments, e.g., toxicity.low, toxicity.moderate, · · · , efficacy.moderate, efficacy.high in the function get.decision.obd.kb(.), here, the investigator is only required to provide the DLT target rate, efficacy target rate, the cohort size, total number of cohorts to automatically generate the boundary and decision table and meanwhile provide , which are generated based on both toxicity and efficacy dose escalation and de-escalation rules.
To generate a default parameter set of toxicity.low, toxicity.moderate, · · · , efficacy.moderate, efficacy.high, here, we used the Bayesian optimal interval design (BOIN) as a tool to automatically divide the toxicity and efficacy intervals. To be specific, given the toxicity/DLT target rate, the BOIN algorithm will produce two cutoffs to divide the toxicity interval into toxicity.low, toxicity-acceptable, toxicity-high sub-intervals.
For example, if the toxicity target rate is 0.2, by executing the following code, the toxicity interval can be divided into three sub-intervsls, (0, 0. 16 For getting the sub-intervals for the efficacy, we would firstly input the efficacy failure rate to the above get.boundary(.) function to get the two cutoffs. The efficacy failure rate is defined as 1-efficacy.target.rate. Then, we use 1 minus the two computed cutoffs and reverse to increasing order to find the sub-intervals. The reason we take this approach is that the BOIN algorithm was originally proposed to find the MTD based on the rationale of higher dose indicating more toxicity, which is undesirable while in the setting of the efficacy, higher Keyboard: An R Package Suite for Early-phase Dose-finding Designs the efficacy is desirable. Actually, a similar strategy was also used in Zohar's research [30]. From the above, we can see that by using this default approach, there are three subintervals for toxicity: • The low interval for toxicity is (0, toxicity.lower.boundary).
• The high interval for toxicity is (toxicity.upper.boundary, 1). and, there are three sub-intervals for efficacy: • The low interval for efficacy is (0, efficacy.lower.boundary).

Design and conduct the trial
To design a phase I/II trial for finding the OBD, we need to run the function get.decision.obd2.kb(·) to obtain the boundary table and decision table, which are we need to run the trial. This function has the following arguments: • target.toxicity The target DLT rate.
• target.efficacy The target efficacy rate.
• cohortsize The number of patients in the cohort.
• ncohort The total number of cohorts.

• decision
The pre-specified decisions.
• cutoff.eli.toxicity The cutoff value to eliminate a dose with unacceptably high toxicity for safety. The default value is 0.95.
• cutoff.eli.efficacy The cutoff value to eliminate a dose with unacceptably low efficacy. The default value is 0.3.
As an example, suppose we want to conduct a phase I/II trial with J = 5 dose levels, a target toxicity rate of target.toxicity = 0.2, and a target efficacy rate of target.ef f icacy = 0.4.

Keyboard: An R Package Suite for Early-phase Dose-finding Designs
Patients are treated in cohorts of size 3 and ncohort is 10. Using the default values of cutoff.eli.toxicity, and cutoff.eli.efficacy provided by the function, we can design a trial by running get.decision.obd2.kb(·), as follows: R> decision <-get.decision.obd2.kb(target.toxicity=0.2, + target.efficacy=0.4, cohortsize=3, ncohort=10, + decision=c("E", "E", "S", "S", "S", "S", "D", "D", "D")) • E: escalate to the next dose level j+1, assuming that the current dose level is j and j is not the highest dose level. Otherwise, treat the next cohort of patients at the current dose level j.
• S: stay at the current dose level j (i.e., treat the next cohort of patients at dose level j).
• EUE: escalate to the next higher dose level because of unacceptably low efficacy.
• DUE: deescalate to the previous lower admissible dose level because of unacceptably low efficacy.
• DUT: deescalate to the previous lower admissible dose level because of unacceptably high toxicity.
• Default stopping rule: if P r(p i > p T |D) is too high (the default is 0.95), then any dose higher than i or equal to i is excluded from the trial.
• Default futility rule: if P r(q i > q E |D) is too low (the default value is 0.3), then dose i is excluded from the trial.

Obtain the operating characteristics
The function get.oc.obd2.kb(·) can be used to obtain the operating characteristics of the design. This function takes the following arguments: • target.toxicity The target DLT rate.
• target.efficacy The target efficacy rate.
• ncohort The total number of cohorts.
• cohortsize The number of patients in the cohort.
• n.early The early stopping parameter. If the number of patients treated at the current dose reaches n.early, then we stop the trial and select the MTD based on the observed data. The default value is 100.
• startdose The starting dose level.
• p.true A vector containing the true toxicity probabilities of the investigational dose levels.
• q.true A vector containing the true efficacy probabilities of the investigational dose levels.
• ntrial The total number of trials to be simulated.
• seed The random seed for simulation.
• p1 The cutoff lower limit for safety utility funciton (3): is the function for toxicity probability, and f(q) is the function for efficacy probability.
• p2 The cutoff upper limit for safety utility function .
• cutoff.eli.toxicity The cutoff value to eliminate a dose with unacceptable high toxicity for safety. The default value is 0.95.
• cutoff.eli.efficacy The cutoff value for the futility rule, the acceptable lowest efficacy. The default value is 0.30.
Using the same setting as above and assuming that the true toxicity scenario is p.true

Select the OBD when the trial is completed
The selection of the OBD uses the same function as in section 3.3.3.

Trial examples 4.1. Single-agent phase I trial
Consider a single-agent phase I trial with 5 dose levels, in which the objective is to find the MTD with a target DLT rate of 0.3. The maximum sample size is 30 patients, treated in cohorts with the size of 3. To design and conduct this trial, we first ran function get.boundary.kb(target=0.3, ncohort=10, cohortsize=3), yielding the dose escalation and de-escalation boundaries as shown in Table 3 . The trial started by treating the first cohort of 3 patients at dose level 1, and none of the patients had DLT. According to the dose escalation and de-escalation rule provided in Table 3, we escalated the dose to level 2 to treat the second cohort of 3 patients; none of whom experienced DLT. Thus, we escalated the dose to level 3 and treated the third cohort of patients, 2 of whom experienced DLT. On the basis of Table 3, we de-escalated the dose back to level 2 and treated the fourth cohort of patients, one of whom experienced DLT.
We then escalated the dose to level 3 and treated the fifth cohort of patients, none of whom experienced DLT. Therefore, the sixth cohort was also treated at dose level 3. Figure 3 shows the dose assignment path for all 30 patients.
At the end of the trial, the number of patients and the number of DLTs at the 5 doses were n=c(3, 6, 18, 3, 0) and y=c(0, 1, 5, 3, 0), respectively. We called function select.mtd.kb(target=0.3, ntox=y, npts=n), which recommended dose level 3 as the MTD, with the estimated DLT rate is 28.0% and the 95% CI =is (10%, 50%). where n records the number of patients treated at each dose combination, and y records the number of patients who experienced DLT at each dose combination. In matrices y and n, entry (j, k) records the data associated with combination (j, k). To determine the dose for the second cohort of patients, we called function next.comb.kb(target=0.3, npts=n, ntox=y, dose.curr=c(1, 1)), which recommended escalating the dose to combination (2,1).

Drug-combination trial to find a single MTD
Therefore, we treated the second cohort of patients at dose combination (2,1). In the second q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q  To determine the dose for the third cohort of patients, we again call next.comb.kb(target=0.3, npts=n, ntox=y, dose.curr=c(2,1)) with updated y, n and dose.curr. The function recommended escalating the dose to (3,1) for treating the third cohort of patients. We repeated this procedure until the maximum sample size was reached. Figure   No toxicity Toxicity Figure 4: Illustration of the dual-agent combination design of finding the MTD for a 3 × 5 combination trial with a cohort of 3. Open circles indicate patients without toxicity, and solid circles denote patients with toxicity.
n=6,t=1, r=4). The output was "E", so we escalated the dose to level 3 to treat the third cohort of 3 patients, 1 of whom experienced DLT and 3 of whom experienced an efficacy response. We repeat this process until maximum sample size reached or stopping rules are satisfied. The complete dose-transition path is shown in Figure 5.

Conclusion
The Keyboard design, as a model-assisted design, combines the advantages of both the algorithm-based design (simplicity) and the model-assisted design (superior performances). The Keyboard designs are novel early-phase trial designs that can be implemented simply and transparently, similar to the 3+3 design, but yield excellent performance comparable to those of more complicated, model-based designs. The developed Keyboard package provides tools for designing, conducting, and analyzing single-agent, drug-combination, and phase I/II dose-finding clinical trials.