A Relaxation Approach to Feature Selection for Linear Mixed Effects Models

Abstract

Linear Mixed-Effects (LME) models are a fundamental tool for modeling correlated data, including cohort studies, longitudinal data analysis, and meta-analysis. Design and analysis of variable selection methods for LMEs is more difficult than for linear regression because LME models are nonlinear. In this article we propose a novel optimization strategy that enables a wide range of variable selection methods for LMEs using both convex and nonconvex regularizers, including ${\mathcal{l}}_1$, Adaptive-${\mathcal{l}}_1$, SCAD, and ${\mathcal{l}}_0$. The computational framework only requires the proximal operator for each regularizer to be readily computable, and the implementation is available in an open source python package pysr3, consistent with the sklearn standard. The numerical results on simulated data sets indicate that the proposed strategy improves on the state of the art for both accuracy and compute time. The variable selection techniques are also validated on a real example using a data set on bullying victimization. Supplementary materials for this article are available online.

KEYWORDS:

• Feature selection
• Mixed effects models
• Nonconvex optimization

Supplementary Materials

Supplementary materials include open source code used to generate simulated examples and perform all numerical comparisons.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 https://github.com/aksholokhov/pysr3

2 https://github.com/aksholokhov/msr3-paper

3 https://rdrr.io/cran/glmmLasso/man/glmmLasso.html

4 https://rdrr.io/cran/lmmlasso/

5 https://rdrr.io/cran/glmmLasso/man/glmmLasso.html

6 https://rdrr.io/cran/lmmlasso/

7 https://rdrr.io/cran/fence/

8 https://rdrr.io/cran/glmmLasso/man/glmmLasso.html

9 https://rdrr.io/cran/lmmlasso/

10 Institute for Health Metrics and Evaluation (IHME). Bullying Victimization Relative Risk Bundle GBD 2020. Seattle, United States of America (USA), 2021.

11 Available at https://github.com/aksholokhov/pysr3

12 https://scikit-learn.org/stable/developers/develop.html

Additional information

Funding

This work was supported by Bill and Melinda Gates Foundation; U.S. NSF grant DMS-1514559.