Bayesian Multi-Task Variable Selection with an Application to Differential DAG Analysis

References

• Agrawal, R., Uhler, C., and Broderick, T. (2018), “Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models,” in International Conference on Machine Learning, pp. 89–98.
   Google Scholar
• Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017), “Variational Inference: A Review for Statisticians,” Journal of the American Statistical Association, 112, 859–877. DOI: 10.1080/01621459.2017.1285773.
   Web of Science ®Google Scholar
• Bonilla, E. V., Chai, K., and Williams, C. (2007), “Multi-Task Gaussian Process Prediction,” in Advances in Neural Information Processing Systems (Vol. 20).
   Google Scholar
• Carbonetto, P., and Stephens, M. (2012), “Scalable Variational Inference for Bayesian Variable Selection in Regression, and its Accuracy in Genetic Association Studies,” Bayesian Analysis, 7, 73–108. DOI: 10.1214/12-BA703.
   Web of Science ®Google Scholar
• Castelletti, F., La Rocca, L., Peluso, S., Stingo, F. C., and Consonni, G. (2020), “Bayesian Learning of Multiple Directed Networks from Observational Data,” Statistics in Medicine, 39, 4745–4766. DOI: 10.1002/sim.8751.
   PubMed Web of Science ®Google Scholar
• Chen, X., Sun, H., Ellington, C., Xing, E., and Song, L. (2021), “Multi-Task Learning of Order-Consistent Causal Graphs,” in Advances in Neural Information Processing Systems (Vol. 34).
   Google Scholar
• Chickering, D. M. (2002), “Optimal Structure Identification with Greedy Search,” Journal of Machine Learning Research, 3, 507–554.
   Google Scholar
• Danaher, P., Wang, P., and Witten, D. M. (2014), “The Joint Graphical Lasso for Inverse Covariance Estimation Across Multiple Classes,” Journal of the Royal Statistical Society, Series B, 76, 373–397. DOI: 10.1111/rssb.12033.
   Google Scholar
• Ellis, B., and Wong, W. H. (2008), “Learning Causal Bayesian Network Structures from Experimental Data,” Journal of the American Statistical Association, 103, 778–789. DOI: 10.1198/016214508000000193.
   Web of Science ®Google Scholar
• Fiers, M. W. E. J., Minnoye, L., Aibar, S., Bravo González-Blas, C., Kalender Atak, Z., and Aerts, S. (2018), “Mapping Gene Regulatory Networks from Single-Cell Omics Data,” Briefings in Functional Genomics, 17, 246–254. DOI: 10.1093/bfgp/elx046.
   PubMed Web of Science ®Google Scholar
• Fronk, E.-M., and Giudici, P. (2004), “Markov Chain Monte Carlo Model Selection for DAG Models,” Statistical Methods and Applications, 13, 259–273. DOI: 10.1007/s10260-004-0097-z.
   Google Scholar
• George, E. I., and McCulloch, R. E. (1993), “Variable Selection via Gibbs Sampling,” Journal of the American Statistical Association, 88, 881–889. DOI: 10.1080/01621459.1993.10476353.
   Web of Science ®Google Scholar
• Ghoshal, A., Bello, K., and Honorio, J. (2019), “Direct Learning with Guarantees of the Difference DAG between Structural Equation Models,” arXiv preprint arXiv:1906.12024.
   Google Scholar
• Gonçalves, A., Von Zuben, F. J., and Banerjee, A. (2016), “Multi-Task Sparse Structure Learning with Gaussian Copula Models,” The Journal of Machine Learning Research, 17, 1205–1234.
   Web of Science ®Google Scholar
• Guo, S., Zoeter, O., and Archambeau, C. (2011), “Sparse Bayesian Multi-Task Learning,” in Advances in Neural Information Processing Systems (Vol. 24).
   Google Scholar
• Harris, N., and Drton, M. (2013), “PC Algorithm for Nonparanormal Graphical Models,” Journal of Machine Learning Research, 14, 3365–3383.
   Web of Science ®Google Scholar
• Hernández-Lobato, D., Hernández-Lobato, J. M., and Ghahramani, Z. (2015), “A Probabilistic Model for Dirty Multi-Task Feature Selection,” in International Conference on Machine Learning, pp. 1073–1082, PMLR.
   Google Scholar
• Huang, X., Wang, J., and Liang, F. (2016), “A Variational Algorithm for Bayesian Variable Selection,” arXiv preprint arXiv:1602.07640.
   Google Scholar
• Ishwaran, H., and Rao, J. S. (2005), “Spike and Slab Variable Selection: Frequentist and Bayesian Strategies,” The Annals of Statistics, 33, 730–773. DOI: 10.1214/009053604000001147.
   Web of Science ®Google Scholar
• Jalali, A., Sanghavi, S., Ruan, C., and Ravikumar, P. (2010), “A Dirty Model for Multi-Task Learning,” in Advances in Neural Information Processing Systems (Vol. 23).
   Google Scholar
• Jeong, S., and Ghosal, S. (2021), “Unified Bayesian Theory of Sparse Linear Regression with Nuisance Parameters,” Electronic Journal of Statistics, 15, 3040–3111. DOI: 10.1214/21-EJS1855.
   Web of Science ®Google Scholar
• Jiang, W., and Tanner, M. A. (2008), “Gibbs Posterior for Variable Selection in High-Dimensional Classification and Data Mining,” The Annals of Statistics, 36, 2207–2231. DOI: 10.1214/07-AOS547.
   Web of Science ®Google Scholar
• Johnson, V. E., and Rossell, D. (2012), “Bayesian Model Selection in High-Dimensional Settings,” Journal of the American Statistical Association, 107, 649–660. DOI: 10.1080/01621459.2012.682536.
   Web of Science ®Google Scholar
• Kalisch, M., Mächler, M., Colombo, D., Maathuis, M. H., and Bühlmann, P. (2012), “Causal Inference Using Graphical Models with the R Package pcalg,” Journal of Statistical Software, 47, 1–26. DOI: 10.18637/jss.v047.i11.
   Web of Science ®Google Scholar
• Kanehisa, M., Goto, S., Sato, Y., Furumichi, M., and Tanabe, M. (2012), KEGG for Integration and Interpretation of Large-Scale Molecular Data Sets,” Nucleic Acids Research, 40, D109–D114. DOI: 10.1093/nar/gkr988.
   PubMed Web of Science ®Google Scholar
• Koller, D., and Friedman, N. (2009), Probabilistic Graphical Models: Principles and Techniques, Cambridge, MA: MIT Press.
   Google Scholar
• Kuipers, J., and Moffa, G. (2017), “Partition MCMC for Inference on Acyclic Digraphs,” Journal of the American Statistical Association, 112, 282–299. DOI: 10.1080/01621459.2015.1133426.
   Web of Science ®Google Scholar
• Kuipers, J., Suter, P., and Moffa, G. (2022), “Efficient Sampling and Structure Learning of Bayesian Networks,” Journal of Computational and Graphical Statistics, 31, 639–650. DOI: 10.1080/10618600.2021.2020127.
   Web of Science ®Google Scholar
• Lee, K., and Cao, X. (2022), “Bayesian Joint Inference for Multiple Directed Acyclic Graphs,” Journal of Multivariate Analysis, 191, 105003. DOI: 10.1016/j.jmva.2022.105003.
   Web of Science ®Google Scholar
• Li, Y., Liu, D., Li, T., and Zhu, Y. (2020), “Bayesian Differential Analysis of Gene Regulatory Networks Exploiting Genetic Perturbations,” BMC Bioinformatics, 21, 1–13. DOI: 10.1186/s12859-019-3314-3.
   PubMed Web of Science ®Google Scholar
• Liu, J., Sun, W., and Liu, Y. (2019), “Joint Skeleton Estimation of Multiple Directed Acyclic Graphs for Heterogeneous Population,” Biometrics, 75, 36–47. DOI: 10.1111/biom.12941.
   PubMed Web of Science ®Google Scholar
• Lounici, K., Pontil, M., Tsybakov, A. B., and Van De Geer, S. (2009), “Taking Advantage of Sparsity in Multi-Task Learning,” arXiv preprint arXiv:0903.1468.
   Google Scholar
• Lounici, K., Pontil, M., Van De Geer, S., and Tsybakov, A. B. (2011), “Oracle Inequalities and Optimal Inference Under Group Sparsity,” The Annals of Statistics, 39, 2164–2204. DOI: 10.1214/11-AOS896.
   Web of Science ®Google Scholar
• Meinshausen, N., and Bühlmann, P. (2010), “Stability Selection,” Journal of the Royal Statistical Society, Series B, 72, 417–473. DOI: 10.1111/j.1467-9868.2010.00740.x.
   Google Scholar
• Nandy, P., Hauser, A., and Maathuis, M. H. (2018), “High-Dimensional Consistency in Score-based and Hybrid Structure Learning,” The Annals of Statistics, 46, 3151–3183. DOI: 10.1214/17-AOS1654.
   Web of Science ®Google Scholar
• Narisetty, N. N., and He, X. (2014), “Bayesian Variable Selection with Shrinking and Diffusing Priors,” The Annals of Statistics, 42, 789–817. DOI: 10.1214/14-AOS1207.
   Web of Science ®Google Scholar
• Niu, X., Sun, Y., and Sun, J. (2018), “Latent Group Structured Multi-Task Learning,” in 2018 52nd Asilomar Conference on Signals, Systems, and Computers, pp. 850–854, IEEE. DOI: 10.1109/ACSSC.2018.8645435.
   Google Scholar
• Ormerod, J. T., You, C., and Müller, S. (2017), “A Variational Bayes Approach to Variable Selection,” Electronic Journal of Statistics, 11, 3549–3594. DOI: 10.1214/17-EJS1332.
   Web of Science ®Google Scholar
• Oyen, D., and Lane, T. (2012), “Leveraging Domain Knowledge in Multitask Bayesian Network Structure Learning,” in Twenty-Sixth AAAI Conference on Artificial Intelligence.
   Google Scholar
• Peterson, C., Stingo, F. C., and Vannucci, M. (2015), “Bayesian Inference of Multiple Gaussian Graphical Models,” Journal of the American Statistical Association, 110, 159–174. DOI: 10.1080/01621459.2014.896806.
   PubMed Web of Science ®Google Scholar
• Peterson, C. B., and Stingo, F. C. (2021), “Bayesian Estimation of Single and Multiple Graphs,” in Handbook of Bayesian Variable Selection, eds. M. G. Tadesse and M. Vannucci, pp. 327–348, New York: Chapman and Hall/CRC.
   Google Scholar
• Peterson, C. B., Osborne, N., Stingo, F. C., Bourgeat, P., Doecke, J. D., and Vannucci, M. (2020), “Bayesian Modeling of Multiple Structural Connectivity Networks During the Progression of Alzheimer’s Disease,” Biometrics, 76, 1120–1132. DOI: 10.1111/biom.13235.
   PubMed Web of Science ®Google Scholar
• Ray, K., and Szabó, B. (2021), “Variational Bayes for High-Dimensional Linear Regression with Sparse Priors,” Journal of the American Statistical Association, 117, 1270–1281. DOI: 10.1080/01621459.2020.1847121.
   Web of Science ®Google Scholar
• Ray, K., and Szabó, B. (2022), “Variational Bayes for High-Dimensional Linear Regression with Sparse Priors,” Journal of the American Statistical Association, 117, 1270–1281. DOI: 10.1080/01621459.2020.1847121.
   Google Scholar
• Shaddox, E., Peterson, C. B., Stingo, F. C., Hanania, N. A., Cruickshank-Quinn, C., Kechris, K., Bowler, R., and Vannucci, M. (2020), “Bayesian Inference of Networks Across Multiple Sample Groups and Data Types,” Biostatistics, 21, 561–576. DOI: 10.1093/biostatistics/kxy078.
   PubMed Web of Science ®Google Scholar
• Spirtes, P., Glymour, C. N., Scheines, R., and Heckerman, D. (2000), Causation, Prediction, and Search, Cambridge, MA: MIT Press.
   Google Scholar
• Tothill, R. W., Tinker, A. V., George, J., Brown, R., Fox, S. B., Lade, S., Johnson, D. S., Trivett, M. K., Etemadmoghadam, D., Locandro, B., et al. (2008), “Novel Molecular Subtypes of Serous and Endometrioid Ovarian Cancer Linked to Clinical Outcome,” Clinical Cancer Research, 14, 5198–5208. DOI: 10.1158/1078-0432.CCR-08-0196.
   PubMed Web of Science ®Google Scholar
• Van de Sande, B., Flerin, C., Davie, K., De Waegeneer, M., Hulselmans, G., Aibar, S., Seurinck, R., Saelens, W., Cannoodt, R., Rouchon, Q., Verbeiren, T., De Maeyer, D., Reumers, J., Saeys, Y., and Aerts, S. (2020), “A Scalable SCENIC Workflow for Single-Cell Gene Regulatory Network Analysis,” Nature Protocols, 15, 2247–2276. DOI: 10.1038/s41596-020-0336-2.
   PubMed Web of Science ®Google Scholar
• Wang, G., Sarkar, A., Carbonetto, P., and Stephens, M. (2020), “A Simple New Approach to Variable Selection in Regression, with Application to Genetic Fine Mapping,” Journal of the Royal Statistical Society, Series B, 82, 1273–1300. DOI: 10.1111/rssb.12388.
   Google Scholar
• Wang, Y., Segarra, S., and Uhler, C. (2020b), “High-Dimensional Joint Estimation of Multiple Directed Gaussian Graphical Models,” Electronic Journal of Statistics, 14, 2439–2483. DOI: 10.1214/20-EJS1724.
   Web of Science ®Google Scholar
• Yajima, M., Telesca, D., Ji, Y., and Müller, P. (2015), “Detecting Differential Patterns of Interaction in Molecular Pathways,” Biostatistics, 16, 240–251. DOI: 10.1093/biostatistics/kxu054.
   PubMed Web of Science ®Google Scholar
• Yang, Y., Wainwright, M. J., and Jordan, M. I. (2016), “On the Computational Complexity of High-Dimensional Bayesian Variable Selection,” The Annals of Statistics, 44, 2497–2532. DOI: 10.1214/15-AOS1417.
   Web of Science ®Google Scholar
• Zhang, A., Zhang, G., Calhoun, V. D., and Wang, Y.-P. (2020), “Causal Brain Network in Schizophrenia by a Two-Step Bayesian Network Analysis,” in Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications (Vol. 11318), pp. 316–321, SPIE. DOI: 10.1117/12.2549306.
   Google Scholar
• Zhang, Y., and Yang, Q. (2021), “A Survey on Multi-Task Learning,” IEEE Transactions on Knowledge and Data Engineering, 34, 5586–5609. DOI: 10.1109/TKDE.2021.3070203.
   Web of Science ®Google Scholar