Augmenting a Simulation Campaign for Hybrid Computer Model and Field Data Experiments

References

• Arendt, P. D., Apley, D. W., and Chen, W. (2016), “A Preposterior Analysis to Predict Identifiability in the Experimental Calibration of Computer Models,” IIE Transactions, 48, 75–88. DOI: 10.1080/0740817X.2015.1064554.
   Web of Science ®Google Scholar
• Ba, S., and Joseph, V. R. (2018), MaxPro: Maximum Projection Designs, R package version 4.1-2.
   Google Scholar
• Balaram, V. (2019), “Rare Earth Elements: A Review of Applications, Occurrence, Exploration, Analysis, Recycling, and Environmental Impact,” Geoscience Frontiers, 10, 1285–1303. DOI: 10.1016/j.gsf.2018.12.005.
   Web of Science ®Google Scholar
• Bastos, L. S., and O’Hagan, A. (2009), “Diagnostics for Gaussian Process Emulators,” Technometrics, 51, 425–438. DOI: 10.1198/TECH.2009.08019.
   Web of Science ®Google Scholar
• Bayarri, M., Berger, J., and Liu, F. (2009), “Modularization in Bayesian Analysis, with Emphasis on Analysis of Computer Models,” Bayesian Analysis, 4, 119–150. DOI: 10.1214/09-BA404.
   Web of Science ®Google Scholar
• Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H., and Tu, J. (2007), “A Framework for Validation of Computer Models,” Technometrics, 49, 138–154. DOI: 10.1198/004017007000000092.
   Web of Science ®Google Scholar
• Bernstein, D. S. (2009). Matrix Mathematics, Princeton: Princeton University Press.
   Google Scholar
• Binois, M., and Gramacy, R. B. (2021), hetGP: Heteroskedastic Gaussian Process Modeling and Design under Replication, R package version 1.1.5.
   Google Scholar
• Binois, M., Huang, J., Gramacy, R. B., and Ludkovski, M. (2019), “Replication or Exploration? Sequential Design for Stochastic Simulation Experiments,” Technometrics, 61, 7–23. DOI: 10.1080/00401706.2018.1469433.
   Web of Science ®Google Scholar
• Brent, R. P. (2013), Algorithms for Minimization Without Derivatives, North Chelmsford, MA: Courier Corporation.
   Google Scholar
• Brynjarsdottir, J., and O’Hagan, A. (2014), “Learning About Physical Parameters: The Importance of Model Discrepancy,” Inverse Problems, 30, 114007. DOI: 10.1088/0266-5611/30/11/114007.
   Web of Science ®Google Scholar
• Byrd, R., Qiu, P., Nocedal, J., and Zhu, C. (1995), “A Limited Memory Algorithm for Bound Constrained Optimization,” Journal on Scientific Computing, 16, 1190–1208.
   Web of Science ®Google Scholar
• Castillo, A. R., Joseph, V. R., and Kalidindi, S. R. (2019), “Bayesian Sequential Design of Experiments for Extraction of Single-Crystal Material Properties from Spherical Indentation Measurements on Polycrystalline Samples,” JOM, 71, 2671–2679. DOI: 10.1007/s11837-019-03549-x.
   Web of Science ®Google Scholar
• Chen, J., Chen, Z., Zhang, C., and Jeff Wu, C. (2022), “Apik: Active Physics-Informed kriging Model with Partial Differential Equations,” SIAM/ASA Journal on Uncertainty Quantification, 10, 481–506. DOI: 10.1137/20M1389285.
   Web of Science ®Google Scholar
• Cohn, D. (1994), “Neural Network Exploration Using Optimal Experiment Design,” in Advances in Neural Information Processing Systems, pp. 679–686.
   Google Scholar
• Cole, D. A., Christianson, R. B., and Gramacy, R. B. (2021), “Locally Induced Gaussian Processes for Large-Scale Simulation Experiments,” Statistics and Computing, 31, 1–21. DOI: 10.1007/s11222-021-10007-9.
   Web of Science ®Google Scholar
• Fer, I., Kelly, R., Moorcroft, P. R., Richardson, A. D., Cowdery, E. M., and Dietze, M. C. (2018), “Linking Big Models to Big Data: Efficient Ecosystem Model Calibration through Bayesian Model Emulation,” Biogeosciences, 15, 5801–5830. DOI: 10.5194/bg-15-5801-2018.
   Web of Science ®Google Scholar
• Goh, J., Bingham, D., Holloway, J. P., Grosskopf, M. J., Kuranz, C. C., and Rutter, E. (2013), “Prediction and Computer Model Calibration Using Outputs from Multifidelity Simulators,” Technometrics, 55, 501–512. DOI: 10.1080/00401706.2013.838910.
   Web of Science ®Google Scholar
• Goonan, T. G. (2012), “Rare Earth Elements-End Use and Recyclability,” in Rare Earth Elements: Supply, Trade and Use Dynamics, pp. 119–138.
   Google Scholar
• Van Gosen, B. S., Verplanck, P. L., Long, K. R., Gambogi, J., and Seal, R. R. (2014), “The Rare-Earth Elements; Vital to Modern Technologies and Lifestyles,” Fact Sheet - U. S. Geological Survey.
   Google Scholar
• Gramacy, R. B. (2016), “lagp: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in r,” Journal of Statistical Software, 72, 1–46. DOI: 10.18637/jss.v072.i01.
   Web of Science ®Google Scholar
• —–(2020), Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences, Boca Raton, FL: Chapman Hall/CRC. Available at http://bobby.gramacy.com/surrogates/.
   Google Scholar
• Gramacy, R. B., Bingham, D., Holloway, J. P., Grosskopf, M. J., Kuranz, C. C., Rutter, E., Trantham, M., and Drake, R. P. (2015), “Calibrating a Large Computer Experiment Simulating Radiative Shock Hydrodynamics,” Annals of Applied Statistics, 9, 1141–1168.
   Web of Science ®Google Scholar
• Gramacy, R. B., Sauer, A., and Wycoff, N. (2022), “Triangulation Candidates for Bayesian Optimization,” in Advances in Neural Information Processing Systems, eds. A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho.
   Google Scholar
• Gu, M. (2019), “Jointly Robust Prior for Gaussian Stochastic Process in Emulation, Calibration and Variable Selection,” Bayesian Analysis, 14, 857–885. DOI: 10.1214/18-BA1133.
   Web of Science ®Google Scholar
• Gu, M., and Wang, L. (2018), “Scaled Gaussian Stochastic Process for Computer Model Calibration and Prediction,” SIAM/ASA Journal on Uncertainty Quantification, 6, 1555–1583. DOI: 10.1137/17M1159890.
   Web of Science ®Google Scholar
• Gupta, C. K., and Krishnamurthy, N. (1992a), “Extractive Metallurgy of Rare Earths,” International Materials Reviews, 37, 197–248. DOI: 10.1179/imr.1992.37.1.197.
   Web of Science ®Google Scholar
• Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A., and Ryne, R. D. (2004), “Combining Field Data and Computer Simulations for Calibration and Prediction,” SIAM Journal on Scientific Computing, 26, 448–466. DOI: 10.1137/S1064827503426693.
   Web of Science ®Google Scholar
• Huang, J., Gramacy, R. B., Binois, M., and Libraschi, M. (2020), “On-site Surrogates for Large-Scale Calibration,” Applied Stochastic Models in Business and Industry, 36, 283–304. preprint on arXiv:1810.01903. DOI: 10.1002/asmb.2523.
   Web of Science ®Google Scholar
• Johnson, L. R. (2008), “Microcolony and Biofilm Formation as a Survival Strategy for Bacteria,” Journal of Theoretical Biology, 251, 24–34. DOI: 10.1016/j.jtbi.2007.10.039.
   PubMed Web of Science ®Google Scholar
• Johnson, M. E., Moore, L. M., and Ylvisaker, D. (1990), “Minimax and Maximin Distance Designs,” Journal of Statistical Planning and Inference, 26, 131–148. DOI: 10.1016/0378-3758(90)90122-B.
   Web of Science ®Google Scholar
• Jones, D., Schonlau, M., and Welch, W. (1998), “Efficient Global Optimization of Expensive Black-Box Functions,” Journal of Global Optimization, 13, 455–492. DOI: 10.1023/A:1008306431147.
   Web of Science ®Google Scholar
• Joseph, V. R., Gul, E., and Ba, S. (2015), “Maximum Projection Designs for Computer Experiments,” Biometrika, 102, 371–380. DOI: 10.1093/biomet/asv002.
   Web of Science ®Google Scholar
• Kennedy, M. C., and O’Hagan, A. (2001), “Bayesian Calibration of Computer Models,” Journal of the Royal Statistical Society, Series B, 63, 425–464. DOI: 10.1111/1467-9868.00294.
   Web of Science ®Google Scholar
• Krishna, A., Joseph, V. R., Ba, S., Brenneman, W. A., and Myers, W. R. (2021), “Robust Experimental Designs for Model Calibration,” Journal of Quality Technology, 54, 441–452. DOI: 10.1080/00224065.2021.1930618.
   Web of Science ®Google Scholar
• Leatherman, E. R., Dean, A. M., and Santner, T. J. (2017), “Designing Combined Physical and Computer Experiments to Maximize Prediction Accuracy,” Computational Statistics & Data Analysis, 113, 346–362. DOI: 10.1016/j.csda.2016.07.013.
   Web of Science ®Google Scholar
• Leatherman, E. R., Santner, T. J., and Dean, A. M. (2018), “Computer Experiment Designs for Accurate Prediction,” Statistics and Computing, 28, 739–751.
   Web of Science ®Google Scholar
• MacKay, D. J. (1992), “Information-based Objective Functions for Active Data Selection,” Neural Computation, 4, 590–604. DOI: 10.1162/neco.1992.4.4.590.
   Web of Science ®Google Scholar
• Marrel, A., Iooss, B., Laurent, B., and Roustant, O. (2009), “Calculations of Sobol Indices for the Gaussian Process Metamodel,” Reliability Engineering & System Safety, 94, 742–751. DOI: 10.1016/j.ress.2008.07.008.
   Web of Science ®Google Scholar
• Matheron, G. (1963), “Principles of Geostatistics,” Economic Geology, 58, 1246–1266. DOI: 10.2113/gsecongeo.58.8.1246.
   Google Scholar
• McKay, M. D., Beckman, R. J., and Conover, W. J. (2000), “A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code,” Technometrics, 42, 55–61. DOI: 10.1080/00401706.2000.10485979.
   Web of Science ®Google Scholar
• Morris, M. D. (2015), “Physical Experimental Design in Support of Computer Model Development,” Technometrics, 57, 45–53. DOI: 10.1080/00401706.2013.879265.
   Web of Science ®Google Scholar
• Nelder, J. A., and Mead, R. (1965), “A Simplex Method for Function Minimization,” The Computer Journal, 7, 308–313. DOI: 10.1093/comjnl/7.4.308.
   Web of Science ®Google Scholar
• Plumlee, M. (2017), “Bayesian Calibration of Inexact Computer Models,” Journal of the American Statistical Association, 112, 1274–1285. DOI: 10.1080/01621459.2016.1211016.
   Web of Science ®Google Scholar
• —–(2019), “Computer Model Calibration with Confidence and Consistency,” Journal of the Royal Statistical Society, Series B, 81, 519–545.
   Google Scholar
• Ranjan, P., Lu, W., Bingham, D., Reese, S., Williams, B. J., Chou, C.-C., Doss, F., Grosskopf, M., and Holloway, J. P. (2011), “Follow-Up Experimental Designs for Computer Models and Physical Processes,” Journal of Statistical Theory and Practice, 5, 119–136. DOI: 10.1080/15598608.2011.10412055.
   Google Scholar
• Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989), “Design and Analysis of Computer Experiments,” Statistical Science, 4, 409–423. DOI: 10.1214/ss/1177012413.
   Google Scholar
• Santner, T., Williams, B., and Notz, W. (2018), The Design and Analysis of Computer Experiments (2nd ed.), New York: Springer–Verlag.
   Google Scholar
• Sauer, A., Gramacy, R. B., and Higdon, D. (2022), “Active Learning for Deep Gaussian Process Surrogates,” Technometrics, 65, 4–18. DOI: 10.1080/00401706.2021.2008505.
   Web of Science ®Google Scholar
• Seo, S., Wallat, M., Graepel, T., and Obermayer, K. (2000), “Gaussian Process Regression: Active Data Selection and Test Point Rejection,” in Mustererkennung 2000, pp. 27–34, Berlin: Springer.
   Google Scholar
• Stein, M. L. (1999), Interpolation of Spatial Data: Some Theory for kriging, New York: Springer.
   Google Scholar
• Sürer, Ö., Plumlee, M., and Wild, S. M. (2023), “Sequential Bayesian Experimental Design for Calibration of Expensive Simulation Models,” arXiv preprint arXiv:2305.16506.
   Google Scholar
• Tuo, R., and Wu, C. F. J. (2015), “Efficient Calibration for Imperfect Computer Models,” Annals of Statistics, 43, 2331–2352.
   Web of Science ®Google Scholar
• Tuo, R., and Wu, C. F. J. (2016), “A Theoretical Framework for Calibration in Computer Models: Parameterization, Estimation and Convergence Properties,” Journal of Uncertainty Quantification, 4, 767–795.
   Web of Science ®Google Scholar
• Ver Hoef, J. M., and Barry, R. P. (1998), “Constructing and Fitting Models for Cokriging and Multivariable Spatial Prediction,” Journal of Statistical Planning and Inference, 69, 275–294. DOI: 10.1016/S0378-3758(97)00162-6.
   Web of Science ®Google Scholar
• Wei, K., Iyer, R., and Bilmes, J. (2015), “Submodularity in Data Subset Selection and Active Learning,” in International Conference on Machine Learning, pp. 1954–1963, PMLR.
   Google Scholar
• Williams, B. J., Loeppky, J. L., Moore, L. M., and Macklem, M. S. (2011), “Batch Sequential Design to Achieve Predictive Maturity with Calibrated Computer Models,” Reliability Engineering & System Safety, 96, 1208–1219. DOI: 10.1016/j.ress.2010.04.017.
   Web of Science ®Google Scholar
• Williams, C. K., and Rasmussen, C. E. (2006), Gaussian Processes for Machine Learning (Vol. 2), Cambridge, MA: MIT Press.
   Google Scholar
• Wong, R. K. W., Storlie, C. B., and Lee, T. C. M. (2017), “A Frequentist Approach to Computer Model Calibration,” Journal of the Royal Statistical Society, Series B, 79, 635–648. DOI: 10.1111/rssb.12182.
   Google Scholar
• Wycoff, N., Binois, M., and Wild, S. M. (2021), “Sequential Learning of Active Subspaces,” Journal of Computational and Graphical Statistics, 30, 1224–1237. DOI: 10.1080/10618600.2021.1874962.
   Web of Science ®Google Scholar