I-optimal or G-optimal: Do we have to choose?

Abstract

When optimizing an experimental design for good prediction performance based on an assumed second order response surface model, it is common to focus on a single optimality criterion, either G-optimality, for best worst-case prediction precision, or I-optimality, for best average prediction precision. In this article, we illustrate how using particle swarm optimization to construct a Pareto front of non-dominated designs that balance these two criteria yields some highly desirable results. In most scenarios, there are designs that simultaneously perform well for both criteria. Seeing alternative designs that vary how they balance the performance of G- and I-efficiency provides experimenters with choices that allow selection of a better match for their study objectives. We provide an extensive repository of Pareto fronts with designs for 17 common experimental scenarios for 2 (design size N = 6 to 12), 3 (N = 10 to 16) and 4 (N = 15, 17, 20) experimental factors. These, when combined with a detailed strategy for how to efficiently analyze, assess, and select between alternatives, provide the reader with the tools to select the ideal design with a tailored balance between G- and I-optimality for their own experimental situations.

Keywords:

• continuous designs
• desirability function
• exact designs
• G-optimality
• I-optimality
• optimal experimental design
• particle swarm optimization
• prediction variance
• Pareto front

Acknowledgments

We first thank Quality Engineering for an expedient, professional, and quality review. We also thank the two anonymous referees whose comments led to substantial improvements to the manuscript.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Stephen J. Walsh

Dr. Stephen J. Walsh is a faculty member in the Department of Mathematics and Statistics at Utah State University. He has over a decade of experience practicing as a quality assurance and experimental design expert in laboratory environments. From 2007 to 2011 he was a researcher at Pacific Northwest National Laboratory in Richland, WA. From 2011 to 2017 he worked as a government Statistician and Quality Manager at the International Atomic Energy Agency in Vienna, Austria. His Ph.D., earned in 2021, provided an adaptation of the Particle Swarm Optimization to solving several difficult optimal design problems. His current research program focuses on the synergistic research bridge between design of experiments and machine learning algorithms.

Lu Lu

Lu Lu is an Associate Professor of Statistics in the Department of Mathematics and Statistics at the University of South Florida in Tampa. She was a postdoctoral research associate in the Statistics Sciences Group at Los Alamos National Laboratory. Her research areas include statistical engineering, reliability analysis, design of experiments, response surface methodology, survey sampling, and multiple objective/response optimization.

Christine M. Anderson-Cook

Christine M. Anderson-Cook is a statistician and Retired Guest Scientist in the Statistical Sciences Group at Los Alamos National Laboratory. Her research areas include statistical engineering, reliability, design of experiments, multiple criterion optimization, and response surface methodology. She is a Fellow of the American Statistical Association (ASA) and the American Society for Quality (ASQ).