A Global-Local Approximation Framework for Large-Scale Gaussian Process Modeling

References

• Ba, S., and Joseph, V. R. (2012), “Composite Gaussian Process Models for Emulating Expensive Functions,” The Annals of Applied Statistics, 6, 1838–1860. DOI: 10.1214/12-AOAS570.
   Web of Science ®Google Scholar
• Basak, S., Petit, S., Bect, J., and Vazquez, E. (2022), “Numerical Issues in Maximum Likelihood Parameter Estimation for Gaussian Process Interpolation,” in Machine Learning, Optimization, and Data Science: 7th International Conference, LOD 2021, Grasmere, UK, October 4–8, 2021, Revised Selected Papers, Part II, pp. 116–131, Springer.
   Google Scholar
• Chalupka, K., Williams, C. K. I., and Murray, I. (2013), “A Framework for Evaluating Approximation Methods for Gaussian Process Regression,” Journal of Machine Learning Research, 14, 333–350.
   Web of Science ®Google Scholar
• Chen, T., and Ren, J. (2009), “Bagging for Gaussian Process Regression,” Neurocomputing, 72, 1605–1610. DOI: 10.1016/j.neucom.2008.09.002.
   Web of Science ®Google Scholar
• Cole, D. A., Christianson, R. B., and Gramacy, R. B. (2021), “Locally Induced Gaussian Processes for Large-Scale Simulation Experiments,” Statistics and Computing, 31, 1–21. DOI: 10.1007/s11222-021-10007-9.
   Web of Science ®Google Scholar
• Dancik, G. M., and Dorman, K. S. (2008), “mlegp: Statistical Analysis for Computer Models of Biological Systems Using R,” Bioinformatics, 24, 1966–1967. DOI: 10.1093/bioinformatics/btn329.
   PubMed Web of Science ®Google Scholar
• Das, K., and Srivastava, A. N. (2010), “Block-GP: Scalable Gaussian Process Regression for Multimodal Data,” in 2010 IEEE International Conference on Data Mining, pp. 791–796.
   Google Scholar
• Deisenroth, M., and Ng, J. W. (2015), “Distributed Gaussian Processes,” in International Conference on Machine Learning, pp. 1481–1490, PMLR.
   Google Scholar
• Dette, H., and Pepelyshev, A. (2010), “Generalized Latin Hypercube Design for Computer Experiments,” Technometrics, 52, 421–429. DOI: 10.1198/TECH.2010.09157.
   Web of Science ®Google Scholar
• Emery, X. (2009), “The kriging Update Equations and their Application to the Selection of Neighboring Data,” Computational Geosciences, 13, 269–280. DOI: 10.1007/s10596-008-9116-8.
   Web of Science ®Google Scholar
• Fasshauer, G. E. (2007), Meshfree Approximation Methods with MATLAB, New Jersey, NJ: World Scientific.
   Google Scholar
• Gardner, J. R., Pleiss, G., Bindel, D., Weinberger, K. Q., and Wilson, A. G. (2018), “GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration,” in Advances in Neural Information Processing Systems.
   Google Scholar
• Gneiting, T. (2002), “Compactly Supported Correlation Functions,” Journal of Multivariate Analysis, 83, 493–508. DOI: 10.1006/jmva.2001.2056.
   Web of Science ®Google Scholar
• Gramacy, R. B. (2016), “laGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R,” Journal of Statistical Software, 72, 1–46. DOI: 10.18637/jss.v072.i01.
   Web of Science ®Google Scholar
• ———(2020), Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences, Boca Raton, FL: Chapman Hall/CRC.
   Google Scholar
• Gramacy, R. B., and Apley, D. W. (2015), “Local Gaussian Process Approximation for Large Computer Experiments,” Journal of Computational and Graphical Statistics, 24, 561–578. DOI: 10.1080/10618600.2014.914442.
   Web of Science ®Google Scholar
• Gramacy, R. B., and Lee, H. K. (2012), “Cases for the Nugget in Modeling Computer Experiments,” Statistics and Computing, 22, 713–722. DOI: 10.1007/s11222-010-9224-x.
   Web of Science ®Google Scholar
• Gramacy, R. B., and Lee, H. K. H. (2008), “Bayesian Treed Gaussian Process Models with An Application to Computer Modeling,” Journal of the American Statistical Association, 103, 1119–1130. DOI: 10.1198/016214508000000689.
   Web of Science ®Google Scholar
• Harari, O., and Steinberg, D. M. (2014), “Convex Combination of Gaussian Processes for Bayesian Analysis of Deterministic Computer Experiments,” Technometrics, 56, 443–454. DOI: 10.1080/00401706.2013.861629.
   Web of Science ®Google Scholar
• Hensman, J., Matthews, A., and Ghahramani, Z. (2015), “Scalable Variational Gaussian Process Classification,” in Artificial Intelligence and Statistics, pp. 351–360, PMLR.
   Google Scholar
• Johnson, S. G. (2007), “The NLopt Nonlinear-Optimization Package,” available at https://github.com/stevengj/nlopt.
   Google Scholar
• Joseph, V. R., and Vakayil, A. (2022), “SPlit: An Optimal Method for Data Splitting,” Technometrics, 64, 166–176. DOI: 10.1080/00401706.2021.1921037.
   Web of Science ®Google Scholar
• Kaufman, C. G., Bingham, D., Habib, S., Heitmann, K., and Frieman, J. A. (2011), “Efficient Emulators of Computer Experiments Using Compactly Supported Correlation Functions, with an Application to Cosmology,” The Annals of Applied Statistics, 5, 2470–2492. DOI: 10.1214/11-AOAS489.
   Web of Science ®Google Scholar
• Keerthi, S., and Chu, W. (2005), “A Matching Pursuit Approach to Sparse Gaussian Process Regression,” in Advances in Neural Information Processing Systems (Vol. 18), Cambridge, MA: MIT Press.
   Google Scholar
• Kenett, R. S., and Zacks, S. (2021), Modern Industrial Statistics: With Applications in R, MINITAB, and JMP, Hoboken, NJ: Wiley.
   Google Scholar
• Lawrence, N., Seeger, M., and Herbrich, R. (2002), “Fast Sparse Gaussian Process Methods: The Informative Vector Machine,” in Advances in Neural Information Processing Systems (Vol. 15), MIT Press.
   Google Scholar
• Lin, L., Bingham, D., Broekgaarden, F., and Mandel, I. (2021), “Uncertainty Quantification of a Computer Model for Binary Black Hole Formation,” The Annals of Applied Statistics, 15, 1604–1627. DOI: 10.1214/21-AOAS1484.
   Web of Science ®Google Scholar
• Mak, S., and Joseph, V. R. (2018), “Support Points,” The Annals of Statistics, 46, 2562–2592. DOI: 10.1214/17-AOS1629.
   Web of Science ®Google Scholar
• Meng, Q., and Ng, S. H. (2015), “An Additive Global and Local Gaussian Process Model for Large Data Sets,” in 2015 Winter Simulation Conference (WSC), pp. 505–516. DOI: 10.1109/WSC.2015.7408191.
   Google Scholar
• Morris, M. D., Mitchell, T. J., and Ylvisaker, D. (1993), “Bayesian Design and Analysis of Computer Experiments: Use of Derivatives in Surface Prediction,” Technometrics, 35, 243–255. DOI: 10.1080/00401706.1993.10485320.
   Web of Science ®Google Scholar
• Park, C., and Apley, D. (2018), “Patchwork kriging for Large-Scale Gaussian Process Regression,” The Journal of Machine Learning Research, 19, 269–311.
   Web of Science ®Google Scholar
• Park, C., and Huang, J. Z. (2016), “Efficient Computation of Gaussian Process Regression for Large Spatial Data Sets by Patching Local Gaussian Processes,” Journal of Machine Learning Research, 17, 1–29.
   Web of Science ®Google Scholar
• Quiñonero-Candela, J., and Rasmussen, C. E. (2005), “A Unifying View of Sparse Approximate Gaussian Process Regression,” Journal of Machine Learning Research, 6, 1939–1959.
   Web of Science ®Google Scholar
• Rasmussen, C., and Ghahramani, Z. (2001), “Infinite Mixtures of Gaussian Process Experts,” in Advances in Neural Information Processing Systems (Vol. 14), Cambridge, MA: MIT Press.
   Google Scholar
• Rasmussen, C. E., and Williams, C. K. I. (2005), Gaussian Processes for Machine Learning, Cambridge, MA: The MIT Press.
   Google Scholar
• Rowan, T. H. (1990), “Functional Stability Analysis of Numerical Algorithms,” PhD thesis, Department of Computer Science, University of Texas at Austin, Austin, TX.
   Google Scholar
• Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989), “Design and Analysis of Computer Experiments,” Statistical Science, 4, 409–423. DOI: 10.1214/ss/1177012413.
   Google Scholar
• Santner, T. J., Williams, B. J., and Notz, W. I. (2003), The Design and Analysis of Computer Experiments, Springer series in statistics. New York: Springer-Verlag.
   Google Scholar
• Snelson, E., and Ghahramani, Z. (2005), “Sparse Gaussian Processes Using Pseudo-Inputs,” in Advances in Neural Information Processing Systems (Vol. 18), MIT Press.
   Google Scholar
• ———(2007), “Local and Global Sparse Gaussian Process Approximations,” in Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, volume 2 of Proceedings of Machine Learning Research, pp. 524–531, San Juan, Puerto Rico. PMLR.
   Google Scholar
• Székely, G. J., and Rizzo, M. L. (2013), “Energy Statistics: A Class of Statistics based on Distances,” Journal of Statistical Planning and Inference, 143, 1249–1272. DOI: 10.1016/j.jspi.2013.03.018.
   Web of Science ®Google Scholar
• Titsias, M. (2009), “Variational Learning of Inducing Variables in Sparse Gaussian Processes,” in Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, volume 5 of Proceedings of Machine Learning Research, eds. D. van Dyk and M. Welling, pp. 567–574, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA. PMLR.
   Google Scholar
• Tresp, V. (2000), “A Bayesian Committee Machine,” Neural Computation, 12, 2719–2741. DOI: 10.1162/089976600300014908.
   PubMed Web of Science ®Google Scholar
• Vakayil, A., and Joseph, V. R. (2022), “Data Twinning,” Statistical Analysis and Data Mining: The ASA Data Science Journal, 15, 598–610. DOI: 10.1002/sam.11574.
   Web of Science ®Google Scholar
• ———(2023), “twingp: A Fast Global-Local Gaussian Process Approximation,” available at https://github.com/avkl/twingp.
   Google Scholar
• Vanhatalo, J., and Vehtari, A. (2008), “Modelling Local and Global Phenomena with Sparse Gaussian Processes,” in Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI’08, pp. 571–578, Arlington, VA: AUAI Press.
   Google Scholar
• Vijayakumar, S., and Schaal, S. (2000), “Locally Weighted Projection Regression: An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space,” in Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000) (Vol. 1), pp. 288–293, Morgan Kaufmann.
   Google Scholar
• Wendland, H. (2004). Scattered Data Approximation. Cambridge Monographs on Applied and Computational Mathematics. Cambridge, UK: Cambridge University Press.
   Google Scholar