Likelihood-Free Parameter Estimation with Neural Bayes Estimators

Abstract

Neural Bayes estimators are neural networks that approximate Bayes estimators. They are fast, likelihood-free, and amenable to rapid bootstrap-based uncertainty quantification. In this article, we aim to increase the awareness of statisticians to this relatively new inferential tool, and to facilitate its adoption by providing user-friendly open-source software. We also give attention to the ubiquitous problem of estimating parameters from replicated data, which we address using permutation-invariant neural networks. Through extensive simulation studies we demonstrate that neural Bayes estimators can be used to quickly estimate parameters in weakly identified and highly parameterized models with relative ease. We illustrate their applicability through an analysis of extreme sea-surface temperature in the Red Sea where, after training, we obtain parameter estimates and bootstrap-based confidence intervals from hundreds of spatial fields in a fraction of a second.

Keywords:

• Amortized inference
• Deep learning
• Exchangeable data
• Extreme-value model
• Permutation invariant
• Point estimation
• Spatial statistics

Acknowledgments

The authors would like to thank Yi Cao for technical support. Thanks also go to Emma Simpson, Jennifer Wadsworth, and Thomas Opitz for providing access to the preprocessed Red Sea dataset. The authors are also grateful to Noel Cressie and Jonathan Rougier for providing comments on the manuscript. We are also grateful to the reviewers and the editors for their helpful comments and suggestions that improved the quality of the manuscript.

Disclosure Statement

The authors report that there are no competing interests to declare.

Additional information

Funding

Matthew Sainsbury-Dale’s and Andrew Zammit-Mangion’s research was supported by an Australian Research Council (ARC) Discovery Early Career Research Award, DE180100203. Matthew Sainsbury-Dale’s research was also supported by an Australian Government Research Training Program Scholarship and a 2022 Statistical Society of Australia (SSA) top-up scholarship. Raphaël Huser was partially supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-CRG2020-4394. The authors would like to acknowledge Johan Barthelemy, NVIDIA, SMART Infrastructure Facility of the University of Wollongong, as well as a 2021 Major Equipment Grant from the University of Wollongong’s University Research Committee, that together have provided access to extensive GPU computing resources.