Structured Shrinkage Priors

Abstract

In many regression settings the unknown coefficients may have some known structure, for instance they may be ordered in space or correspond to a vectorized matrix or tensor. At the same time, the unknown coefficients may be sparse, with many nearly or exactly equal to zero. However, many commonly used priors and corresponding penalties for coefficients do not encourage simultaneously structured and sparse estimates. In this article we develop structured shrinkage priors that generalize multivariate normal, Laplace, exponential power and normal-gamma priors. These priors allow the regression coefficients to be correlated a priori without sacrificing elementwise sparsity or shrinkage. The primary challenges in working with these structured shrinkage priors are computational, as the corresponding penalties are intractable integrals and the full conditional distributions that are needed to approximate the posterior mode or simulate from the posterior distribution may be nonstandard. We overcome these issues using a flexible elliptical slice sampling procedure, and demonstrate that these priors can be used to introduce structure while preserving sparsity. Supplementary materials for this article are available online.

Keywords:

• Bayesian lasso
• Lasso
• Multivariate Laplace
• Multivariate normal-scale mixture
• Shrinkage
• Sparsity

Supplementary Materials

Supplementary material available online includes proofs of all the propositions and additional numerical results. A stand-alone package for implementing the methods described in this article can be downloaded from https://github.com/maryclare/sspcomp.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was partially supported by NSF grants DGE-1256082 and DMS-1505136.