Bayesian Heterogeneous Hidden Markov Models with an Unknown Number of States

Abstract

Hidden Markov models (HMMs) are valuable tools for analyzing longitudinal data due to their capability to describe dynamic heterogeneity. Conventional HMMs typically assume that the number of hidden states (i.e., the order of HMMs) is known or predetermined through criterion-based methods. However, prior knowledge about the order is often unavailable, and a pairwise comparison using criterion-based methods becomes increasingly tedious and computationally demanding when the model space enlarges. A few studies have considered simultaneously performing order selection and parameter estimation under the frequentist framework. Still, they focused only on homogeneous HMMs and thus cannot accommodate situations where potential covariates affect the between-state transition. This study proposes a Bayesian double-penalized (BDP) procedure to conduct a simultaneous order selection and parameter estimation for heterogeneous HMMs. We develop a novel Markov chain Monte Carlo algorithm coupled with an efficient adjust-bound reversible jump scheme to address the challenges in updating the order. Simulation studies show that the proposed BDP procedure considerably outperforms the commonly used criterion-based methods. An application to the Alzheimer’s Disease Neuroimaging Initiative study further confirms the utility of the proposed method. Supplementary materials for this article are available online.

KEYWORDS:

• Bayesian method
• Double penalization
• Dynamic heterogeneity
• Longitudinal data
• Order selection

Supplementary Materials

Supplementary Material: The supplemental files include the Appendix which gives the proof of the propositions in Section 3, full conditional distributions in Section 4, and additional numerical results in Sections 5 and 6. (BHMM Supp.pdf)

R code: The supplemental files for this article include R programs which can be used to replicate the simulation study included in the article. Please read file README contained in the zip file for more details. (program package.zip)

Acknowledgments

The authors are thankful to the editor, the associate editor, and two anonymous reviewers for their valuable comments and suggestions, which have helped improve the article substantially.

Disclosure Statement

The authors report there are no competing interest to declare.

Additional information

Funding

This research was fully supported by GRF Grants (14302519, 14302220) from Research Grant Council of the Hong Kong Special Administration Region.