Improved Pathwise Coordinate Descent for Power Penalties

Abstract

Pathwise coordinate descent algorithms have been used to compute entire solution paths for lasso and other penalized regression problems quickly with great success. They improve upon cold start algorithms by solving the problems that make up the solution path sequentially for an ordered set of tuning parameter values, instead of solving each problem separately. However, extending pathwise coordinate descent algorithms to more the general bridge or power family of ${\mathcal{l}}_q$ penalties is challenging. Faster algorithms for computing solution paths for these penalties are needed because ${\mathcal{l}}_q$ penalized regression problems can be nonconvex and especially burdensome to solve. In this article, we show that a reparameterization of ${\mathcal{l}}_q$ penalized regression problems is more amenable to pathwise coordinate descent algorithms. This allows us to improve computation of the mode-thresholding function for ${\mathcal{l}}_q$ penalized regression problems in practice and introduce two separate pathwise algorithms. We show that either pathwise algorithm is faster than the corresponding cold start alternative, and demonstrate that different pathwise algorithms may be more likely to reach better solutions. Supplemental materials for this article are available online.

KEYWORDS:

• Coordinate descent
• LASSO
• Nonconvex optimization
• Regularization surface
• Sparse regression

Supplementary Materials

Appendix: Proofs and additional comparisons. (appx_powcd.pdf, PDF document)

Code and Data: Code and data for replication. (JCGS_Submission_Code.zip, ZIP archive)

Data Availability Statement

Replication code is available at https://github.com/maryclare/powreg.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

This research was supported by NSF grant DMS-2113079.