Likelihood-Free Parameter Estimation with Neural Bayes Estimators

References

• Banerjee, S., Carlin, B. P., and Gelfand, A. E. (2004), Hierarchical Modeling and Analysis for Spatial Data, Boca Raton, FL: Chapman and Hall/CRC Press.
   Google Scholar
• Banesh, D., Panda, N., Biswas, A., Roekel, L. V., Oyen, D., Urban, N., Grosskopf, M., Wolfe, J., and Lawrence, E. (2021). “Fast Gaussian Process Estimation for Large-Scale in Situ Inference Using Convolutional Neural Networks,” in IEEE International Conference on Big Data, pp. 3731–3739.
   Google Scholar
• Beaumont, M. A., Zhang, W., and Balding, D. J. (2002), “Approximate Bayesian Computation in Population Genetics,” Genetics, 162, 2025–2035. DOI: 10.1093/genetics/162.4.2025.
   PubMed Web of Science ®Google Scholar
• Bevilacqua, M., Gaetan, C., Mateu, J., and Porcu, E. (2012), “Estimating Space and Space-Time Covariance Functions for Large Data Sets: A Weighted Composite Likelihood Approach,” Journal of the American Statistical Association, 107, 268–280. DOI: 10.1080/01621459.2011.646928.
   Web of Science ®Google Scholar
• Bezanson, J., Edelman, A., Karpinski, S., and Shah, V. B. (2017), “Julia: A Fresh Approach to Numerical Computing,” SIAM Review, 59, 65–98. DOI: 10.1137/141000671.
   Web of Science ®Google Scholar
• Casella, G., and Berger, R. (2001), Statistical Inference (2nd ed.), Belmont, CA: Duxbury.
   Google Scholar
• Castruccio, S., Huser, R., and Genton, M. G. (2016), “High-Order Composite Likelihood Inference for Max-Stable Distributions and Processes,” Journal of Computational and Graphical Statistics, 25, 1212–1229. DOI: 10.1080/10618600.2015.1086656.
   Web of Science ®Google Scholar
• Chan, J., Perrone, V., Spence, J., Jenkins, P., Mathieson, S., and Song, Y. (2018), “A Likelihood-Free Inference Framework for Population Genetic Data Using Exchangeable Neural Networks,” in Advances in Neural Information Processing Systems (Vol. 31), eds. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Curran Associates, Inc.
   Google Scholar
• Chon, K., and Cohen, R. (1997), “Linear and Nonlinear ARMA Model Parameter Estimation Using an Artificial Neural Network,” IEEE Transactions on Biomedical Engineering, 44, 168–174. DOI: 10.1109/10.554763.
   PubMed Web of Science ®Google Scholar
• Cox, D., and Reid, N. (2004), “A Note on Pseudolikelihood Constructed from Marginal Densities,” Biometrika, 3, 729–737. DOI: 10.1093/biomet/91.3.729.
   Web of Science ®Google Scholar
• Creel, M. (2017), “Neural Nets for Indirect Inference,” Econometrics and Statistics, 2, 36–49. DOI: 10.1016/j.ecosta.2016.11.008.
   Google Scholar
• Cressie, N. (1993), Statistics for Spatial Data (rev. ed.), Hoboken, NJ: Wiley.
   Google Scholar
• ——- (2018), “Mission co2ntrol: A Statistical Scientist’s Role in Remote Sensing of Atmospheric Carbon Dioxide,” Journal of the American Statistical Association, 113, 152–168.
   Web of Science ®Google Scholar
• ——- (2023), “Decisions, Decisions, Decisions in an Uncertain Environment,” Environmetrics, 34, e2767.
   Web of Science ®Google Scholar
• Davison, A. C., and Huser, R. (2015), “Statistics of Extremes,” Annual Review of Statistics and its Application, 2, 203–235. DOI: 10.1146/annurev-statistics-010814-020133.
   Web of Science ®Google Scholar
• Davison, A. C., Huser, R., and Thibaud, E. (2019), “Spatial Extremes,” in Handbook of Environmental and Ecological Statistics, eds. A. E. Gelfand, M. Fuentes, J. A. Hoeting, and R. L. Smith, pp. 711–744, Boca Raton, FL: Chapman & Hall/CRC Press.
   Google Scholar
• Diggle, P. J., and Gratton, R. J. (1984), “Monte Carlo Methods of Inference for Implicit Statistical Models,” Journal of the Royal Statistical Society, Series B, 46, 193–227. DOI: 10.1111/j.2517-6161.1984.tb01290.x.
   Google Scholar
• Diggle, P. J., and Ribeiro, P. J. (2007), Model-Based Geostatistics, New York, NY: Springer.
   Google Scholar
• Donlon, C. J., Martin, M., Stark, J., Roberts-Jones, J., Fiedler, E., and Wimmer, W. (2012), “The Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) System,” Remote Sensing of Environment, 116, 140–158. DOI: 10.1016/j.rse.2010.10.017.
   Web of Science ®Google Scholar
• Flagel, L., Brandvain, Y., and Schrider, D. R. (2018), “The Unreasonable Effectiveness of Convolutional Neural Networks in Population Genetic Inference,” Molecular Biology and Evolution, 36, 220–238. DOI: 10.1093/molbev/msy224.
   Web of Science ®Google Scholar
• Gaskin, T., Pavliotis, G. A., and Girolami, M. (2023), “Neural Parameter Calibration for Large-Scale Multiagent Models,” Proceedings of the National Academy of Sciences, 120, e2216415120. DOI: 10.1073/pnas.2216415120.
   PubMed Web of Science ®Google Scholar
• Gerber, F., and Nychka, D. W. (2021), “Fast Covariance Parameter Estimation of Spatial Gaussian Process Models Using Neural Networks,” Stat, 10, e382. DOI: 10.1002/sta4.382.
   Web of Science ®Google Scholar
• Goodfellow, I., Bengio, Y., and Courville, A. (2016), Deep Learning, Cambridge, MA: MIT Press.
   Google Scholar
• Han, J., Li, Y., Lin, L., Lu, J., Zhang, J., and Zhang, L. (2022), “Universal Approximation of Symmetric and Anti-Symmetric Functions,” Communications in Mathematical Sciences, 20, 1397–1408. DOI: 10.4310/CMS.2022.v20.n5.a8.
   Web of Science ®Google Scholar
• Hazra, A., and Huser, R. (2021), “Estimating High-Resolution Red Sea Surface Temperature Hotspots, Using a Low-Rank Semiparametric Spatial Model,” Annals of Applied Statistics, 15, 572–596.
   Web of Science ®Google Scholar
• Heffernan, J. E., and Tawn, J. A. (2004), “A Conditional Approach for Multivariate Extreme Values,” Journal of the Royal Statistical Society, Series B, 66, 497–546. DOI: 10.1111/j.1467-9868.2004.02050.x.
   Google Scholar
• Hornik, K., Stinchcombe, M., and White, H. (1989), “Multilayer Feedforward Networks are Universal Approximators,” Neural Networks, 2, 359–366. DOI: 10.1016/0893-6080(89)90020-8.
   Web of Science ®Google Scholar
• Huser, R. (2021), “Editorial: EVA 2019 Data Competition on Spatio-Temporal Prediction of Red Sea Surface Temperature Extremes,” Extremes, 24, 91–104. DOI: 10.1007/s10687-019-00369-9.
   Web of Science ®Google Scholar
• Huser, R., and Davison, A. C. (2013), “Composite Likelihood Estimation for the Brown–Resnick Process,” Biometrika, 100, 511–518. DOI: 10.1093/biomet/ass089.
   Web of Science ®Google Scholar
• Huser, R., Dombry, C., Ribatet, M., and Genton, M. G. (2019), “Full Likelihood Inference for Max-Stable Data,” Stat, 8, e218. DOI: 10.1002/sta4.218.
   Web of Science ®Google Scholar
• Huser, R., Stein, M. L., and Zhong, P. (2022), “Vecchia Likelihood Approximation for Accurate and Fast Inference in Intractable Spatial Extremes Models,” arXiv:2203.05626v1.
   Google Scholar
• Huser, R., and Wadsworth, J. (2022), “Advances in Statistical Modeling of Spatial Extremes,” Wiley Interdisciplinary Reviews: Computational Statistics, 14, e1537.
   Google Scholar
• Innes, M. (2018), “Flux: Elegant Machine Learning with Julia,” Journal of Open Source Software, 3, 602. DOI: 10.21105/joss.00602.
   Google Scholar
• Lehmann, E. L., and Casella, G. (1998), Theory of Point Estimation (2nd ed.), New York: Springer.
   Google Scholar
• Lenzi, A., Bessac, J., Rudi, J., and Stein, M. L. (2023), “Neural Networks for Parameter Estimation in Intractable Models,” Computational Statistics & Data Analysis, 185, 107762. DOI: 10.1016/j.csda.2023.107762.
   Web of Science ®Google Scholar
• Lintusaari, J., Gutmann, M., Dutta, R., Kaski, S., and Corander, J. (2017), “Fundamentals and Recent Developments in Approximate Bayesian Computation,” Systematic Biology, 66, 66–82. DOI: 10.1093/sysbio/syw077.
   PubMed Web of Science ®Google Scholar
• McCullagh, P. (2002), “What is a Statistical Model,” The Annals of Statistics, 30, 1225–1310. DOI: 10.1214/aos/1035844977.
   Web of Science ®Google Scholar
• Murphy, R. L., Srinivasan, B., Rao, V. A., and Ribeiro, B. (2019), “Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs,” in 7th International Conference on Learning Representations, ICLR.
   Google Scholar
• Pacchiardi, L., and Dutta, R. (2022), “Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization,” arXiv:2205.15784.
   Google Scholar
• Padoan, S. A., Ribatet, M., and Sisson, S. A. (2010), “Likelihood-based Inference for Max-Stable Processes,” Journal of the American Statistical Association, 105, 263–277. DOI: 10.1198/jasa.2009.tm08577.
   Web of Science ®Google Scholar
• Radev, S. T., Mertens, U. K., Voss, A., Ardizzone, L., and Köthe, U. (2022), “BayesFlow: Learning Complex Stochastic Models with Invertible Neural Networks,” IEEE Transactions on Neural Networks and Learning Systems, 33, 1452–1466. DOI: 10.1109/TNNLS.2020.3042395.
   PubMed Web of Science ®Google Scholar
• R Core Team. (2023), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing.
   Google Scholar
• Richards, J., Tawn, J. A., and Brown, S. (2022), “Modelling Extremes of Spatial Aggregates of Precipitation Using Conditional Methods,” Annals of Applied Statistics, 16, 2693–2713.
   Web of Science ®Google Scholar
• Rudi, J., Julie, B., and Lenzi, A. (2021), “Parameter Estimation with Dense and Convolutional Neural Networks Applied to the FitzHugh-Nagumo ODE,” in Proceedings of the 2nd Annual Conference on Mathematical and Scientific Machine Learning, volume 145 of Proceedings of Machine Learning Research, eds. J. Bruna, J. Hesthaven, and L. Zdeborova, pp. 1–28, PMLR.
   Google Scholar
• Sang, H., and Genton, M. G. (2012), “Tapered Composite Likelihood for Spatial Max-Stable Models,” Spatial Statistics, 8, 86–103. DOI: 10.1016/j.spasta.2013.07.003.
   Web of Science ®Google Scholar
• Schlather, M. (2002), “Models for Stationary Max-Stable Random Fields,” Extremes, 5, 33–44.
   Google Scholar
• Simpson, E. S., Opitz, T., and Wadsworth, J. L. (2023), “High-Dimensional Modeling of Spatial and Spatio-Temporal Conditional Extremes Using INLA and Gaussian Markov Random Fields,” Extremes, to appear. DOI: 10.1007/s10687-023-00468-8.
   Web of Science ®Google Scholar
• Simpson, E. S., and Wadsworth, J. L. (2021), “Conditional Modelling of Spatio-Temporal Extremes for Red Sea Surface Temperatures,” Spatial Statistics, 41, 100482. DOI: 10.1016/j.spasta.2020.100482.
   Web of Science ®Google Scholar
• Sisson, S. A., Fan, Y., and Beaumont, M. (2018), Handbook of Approximate Bayesian Computation, Boca Raton, FL: Chapman & Hall/CRC Press.
   Google Scholar
• Soelch, M., Akhundov, A., van der Smagt, P., and Bayer, J. (2019), “On Deep Set Learning and the Choice of Aggregations,” in Proceedings of the 28th International Conference on Artificial Neural Networks, ICANN, Lecture Notes in Computer Science, eds. I. V. Tetko, V. Kurková, P. Karpov, and F. J. Theis, pp. 444–457, Springer.
   Google Scholar
• Stein, M. L. (1999), Interpolation of Spatial Data: Some Theory for Kriging, New York: Springer.
   Google Scholar
• Stein, M. L., Chi, Z., and Welty, L. J. (2004), “Approximating Likelihoods for Large Spatial Data Sets,” Journal of the Royal Statistical Society, Series, B, 66, 275–296. DOI: 10.1046/j.1369-7412.2003.05512.x.
   Google Scholar
• Strasser, H. (1981), “Consistency of Maximum Likelihood and Bayes Estimates,” The Annals of Statistics, 9, 1107–1113. DOI: 10.1214/aos/1176345590.
   Web of Science ®Google Scholar
• Subbotin, M. T. (1923), “On the Law of Frequency of Errors,” Mathematicheskii Sbornik, 31, 296–301.
   Google Scholar
• Varin, C., Reid, N., and Firth, D. (2011), “An Overview of Composite Likelihood Methods,” Statistica Sinica, 21, 5–42.
   Web of Science ®Google Scholar
• Varin, C., and Vidoni, P. (2005), “A Note on Composite Likelihood Inference and Model Selection,” Biometrika, 92, 519–528. DOI: 10.1093/biomet/92.3.519.
   Web of Science ®Google Scholar
• Vecchia, A. V. (1988), “Estimation and Model Identification for Continuous Spatial Processes,” Journal of the Royal Statistical Society, Series B, 50, 297–312. DOI: 10.1111/j.2517-6161.1988.tb01729.x.
   Google Scholar
• Wadsworth, J. L., and Tawn, J. A. (2022), “Higher-Dimensional Spatial Extremes via Single-Site Conditioning,” Spatial Statistics, 51, 100677. DOI: 10.1016/j.spasta.2022.100677.
   Web of Science ®Google Scholar
• Wagstaff, E., Fuchs, F. B., Engelcke, M., Osborne, M., and Posner, I. (2022), “Universal Approximation of Functions on Sets,” Journal of Machine Learning Research, 23, 1–56.
   Web of Science ®Google Scholar
• Wagstaff, E., Fuchs, F. B., Engelcke, M., Posner, I., and Osborne, M. (2019), “On the Limitations of Representing Functions on Sets,” in Proceedings of the 36th International Conference on Machine Learning (Vol. 97), eds. K. Chaudhuri and R. Salakhutdinov, pp. 6487–6494, PMLR.
   Google Scholar
• Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., and Yu, P. S. (2021), “A Comprehensive Survey on Graph Neural Networks,” IEEE Transactions on Neural Networks and Learning Systems, 32, 4–24. DOI: 10.1109/TNNLS.2020.2978386.
   PubMed Web of Science ®Google Scholar
• Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J. (2017), “Deep Sets,” in Advances in Neural Information Processing Systems (Vol. 30), eds. I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Curran Associates, Inc.
   Google Scholar
• Zammit-Mangion, A., and Wikle, C. K. (2020), “Deep Integro-Difference Equation Models for Spatio-Temporal Forecasting,” Spatial Statistics, 37, 100408. DOI: 10.1016/j.spasta.2020.100408.
   Web of Science ®Google Scholar
• Zhang, H. (2004), “Inconsistent Estimation and Asymptotically Equal Interpolations in Model-based Geostatistics,” Journal of the American Statistical Association, 99, 250–261. DOI: 10.1198/016214504000000241.
   Web of Science ®Google Scholar
• Zhou, D. (2018), “Universality of Deep Convolutional Neural Networks,” Applied and Computational Harmonic Analysis, 48, 787–794. DOI: 10.1016/j.acha.2019.06.004.
   Web of Science ®Google Scholar