Two important topics relevant to late-stage clinical trials are addressed: quantifying the benefit of an intervention relative to its risk of adverse events (AEs) and handling of missing data. The assessment of whether benefit outweighs risk is subjective. Consider the case of lecanemab. It was authorized under the U.S. Food and Drug Administration’s (FDA) accelerated approval pathway for the treatment of Alzheimer’s disease. The primary endpoint was change from baseline at 18 months in a measure of cognition in patients with early Alzheimer’s disease. Key secondary endpoints were related to amyloid burden and other measures of cognition and function. The results for the primary and key secondary endpoints were statistically significant. However, the rates of amyloid related imaging abnormalities (edema/effusions or hemosiderin deposits), AEs of interest, were higher on lecanemab (21.5%) than on placebo (9.5%)1. What is the process of deciding if the net benefit is positive? How does one weigh the statistically significant result on efficacy endpoints with the higher rates of certain AEs? The criteria are not pre-specified, and the decision-making process is subjective and reactive. We make a proposal to quantify the benefit vs risk tradeoff. It has the additional advantages of making the decision-making step transparent and prospective.
Missing data occurs when patients do not complete their scheduled assessments usually due to early withdrawal. The reasons for premature termination of patient participation are diverse and can comprise the experience of AEs as well as withdrawal of consent or being lost to follow-up. As has been noted, the ‘reliability of results from clinical trials can be substantially reduced by missing data.’2. The following three examples of Phase 3 trials each expose different problems with missing data handling methods.
The first is a trial in sickle cell disease patients with a primary endpoint of recurrent sickle cell crises. The topic of missingness was front and center at the U.S. FDA Advisory Committee meeting. The FDA summary review document stated that “The review team was concerned with the amount of missing data and the imputation methods the Applicant used to overcome the impact of the missing efficacy data.”3 Not only was the missing data rate high, but there was also a higher missingness rate in the active group (36%) compared to placebo (24%). Further, “for the patients who discontinued the trial medication or placebo, the number of pain crises was imputed as either the mean number of crises (rounded to the nearest integer) in patients in the same trial group who completed the trial or the actual number of crises the patient had at the time of discontinuation, whichever was greater.”4 Because of the earlier and higher dropout rate in the active group, such an imputation approach will undercount the pain crises in the active group and is not to be recommended.
The second is the Alzheimer's trial referred to above. The rates of AEs leading to discontinuation of study drug were higher for lecanemab (6.9%) compared to placebo (2.9%). The primary analysis was performed without imputation of missing values. The analysis will be biased in favor of lecanemab because it assumes that patients who discontinue will respond similarly to patients who remain in the study. This is clearly not the case here.
The third is a trial conducted in patients with transthyretin amyloidosis cardiomyopathy5. Although the primary endpoint was a hierarchical combination of time to all-cause mortality and frequency of cardiovascular-related hospitalizations, we draw attention to the first secondary endpoint, change from baseline to month 30 in the 6-minute walk test (6MWT). 41% and 60% of patients randomized to the active and placebo groups, respectively, did not complete the month 30 6MWT assessment. The higher missing month 30 6MWT rate in the placebo group was in part due to a benefit on the primary endpoint. The imputation method should be such that the active group should be given ‘credit’ for a lower rate compared to placebo. There is a risk that the typical imputation methods, which are conservative, can substantially understate the treatment effect because of the higher missingness rate on placebo.
The methods proposed are illustrated using data from the Pulmonary Arterial hyperTENsion sGC stimulator Trial-1 (PATENT-1) and additionally described conceptually for benefit-risk by incorporating patient preferences in renal cell carcinoma for which trial data are unavailable6–7. PATENT-1 was a randomized, double-blind, placebo-controlled phase III trial evaluating riociguat in patients with pulmonary arterial hypertension (PAH). Patients were randomized in a 4:2:1 ratio to riociguat individual dose titration (IDT), placebo and an exploratory lower dose arm of riociguat. The predefined efficacy analyses compared the riociguat IDT and placebo groups and we will follow this for the analyses to be discussed. 254 and 126 patients were randomized to riociguat IDT and placebo, respectively. The primary outcome of the study was change from baseline in 6MWT at week 12. Among the secondary endpoints was time to clinical worsening, a disease-specific composite event endpoint which included events like death, hospitalization for PAH and initiation of new PAH therapy.