Sensitivity Analyses of Clinical Trial Designs: Selecting Scenarios and Summarizing Operating Characteristics

Abstract

The use of simulation-based sensitivity analyses is fundamental for evaluating and comparing candidate designs of future clinical trials. In this context, sensitivity analyses are especially useful to assess the dependence of important design operating characteristics with respect to various unknown parameters. Typical examples of operating characteristics include the likelihood of detecting treatment effects and the average study duration, which depend on parameters that are unknown until after the onset of the clinical study, such as the distributions of the primary outcomes and patient profiles. Two crucial components of sensitivity analyses are (i) the choice of a set of plausible simulation scenarios and (ii) the list of operating characteristics of interest. We propose a new approach for choosing the set of scenarios to be included in a sensitivity analysis. We maximize a utility criterion that formalizes whether a specific set of sensitivity scenarios is adequate to summarize how the operating characteristics of the trial design vary across plausible values of the unknown parameters. Then, we use optimization techniques to select the best set of simulation scenarios (according to the criteria specified by the investigator) to exemplify the operating characteristics of the trial design. We illustrate our proposal in three trial designs. Supplementary materials for this article are available online.

KEYWORDS:

• Clinical trial design
• Function approximation
• Operating characteristics
• Sensitivity analysis
• Simulated annealing

Supplementary Materials

The online supplement includes example code (ExampleCode.R) for implementing ROSA.

Acknowledgments

The authors thank Cyrus Mehta and Christina Howe for helpful conversations and feedback that greatly enhanced the article.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

LH was supported by the Clinical Orthopedic and Musculoskeletal Education and Training (COMET) Program, NIAMS grant T32 AR055885. LT was supported by the NIH grant R01LM013352.