Sequential Bayesian Experimental Design for Calibration of Expensive Simulation Models

Abstract

Simulation models of critical systems often have parameters that need to be calibrated using observed data. For expensive simulation models, calibration is done using an emulator of the simulation model built on simulation output at different parameter settings. Using intelligent and adaptive selection of parameters to build the emulator can drastically improve the efficiency of the calibration process. The article proposes a sequential framework with a novel criterion for parameter selection that targets learning the posterior density of the parameters. The emergent behavior from this criterion is that exploration happens by selecting parameters in uncertain posterior regions while simultaneously exploitation happens by selecting parameters in regions of high posterior density. The advantages of the proposed method are illustrated using several simulation experiments and a nuclear physics reaction model.

KEYWORDS:

• Acquisition
• Active learning
• Emulation
• Gaussian process
• Uncertainty quantification

Supplementary Materials

The supplementary materials contain (i) the comparison of EIVAR with the expected improvement and the integrated mean squared error criteria, (ii) descriptions of test functions, (iii) experiments with the minimum energy design criterion, (iv) experiments to investigate the effect of prior, and (v) the code for the test functions.

Acknowledgments

The authors are grateful to Amy Lovell and Filomena M. Nunes for taking time to provide us physics reaction model. We thank the editor, the associate editor, and two anonymous referees for their valuable feedback for improving this article’s exposition. We gratefully acknowledge the computing resources provided on Bebop, a high-performance computing cluster operated by the Laboratory Computing Resource Center at Argonne National Laboratory.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

All three authors are grateful for support from the National Science Foundation (NSF) grant OAC 2004601. This work was further supported by the NSF grant DMS 1953111 and by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, applied mathematics and SciDAC programs under Contract No. DE-AC02-05CH11231.