Statistically Valid Variational Bayes Algorithm for Ising Model Parameter Estimation

Abstract

Ising models originated in statistical physics and are widely used in modeling spatial data and computer vision problems. However, statistical inference of this model remains challenging due to intractable nature of the normalizing constant in the likelihood. Here, we use a pseudo-likelihood instead, to study the Bayesian estimation of two-parameter, inverse temperature and magnetization, Ising model with a fully specified coupling matrix. We develop a computationally efficient variational Bayes procedure for model estimation. Under the Gaussian mean-field variational family, we derive posterior contraction rates of the variational posterior obtained under the pseudo-likelihood. We also discuss the loss incurred due to variational posterior over true posterior for the pseudo-likelihood approach. Extensive simulation studies validate the efficacy of mean-field Gaussian and bivariate Gaussian families as the possible choices of the variational family for inference of Ising model parameters. Supplementary materials for this article are available online.

Keywords:

• Black box variational inference
• Coupling matrix
• ELBO
• Kullback-Leibler distance
• Posterior contraction rates
• Pseudo-likelihood
• Stochastic Optimization

Supplementary Materials

The supplementary file contains implementation details and theoretical details.

Acknowledgments

We are thankful to the Associate Editor and the Reviewers for their comments which helped improve the manuscript significantly.

Disclosure Statement

There is no conflict of interest to report.

Additional information

Funding

The authors are grateful to P. Ghosal and S. Mukherjee for kindly sharing codes of their work. The research is partially supported by the National Science Foundation grants NSF DMS-1952856 and 1924724.