Approximating Partial Likelihood Estimators via Optimal Subsampling

Abstract

With the growing availability of large-scale biomedical data, it is often time-consuming or infeasible to directly perform traditional statistical analysis with relatively limited computing resources at hand. We propose a fast subsampling method to effectively approximate the full data maximum partial likelihood estimator in Cox’s model, which largely reduces the computational burden when analyzing massive survival data. We establish consistency and asymptotic normality of a general subsample-based estimator. The optimal subsampling probabilities with explicit expressions are determined via minimizing the trace of the asymptotic variance-covariance matrix for a linearly transformed parameter estimator. We propose a two-step subsampling algorithm for practical implementation, which has a significant reduction in computing time compared to the full data method. The asymptotic properties of the resulting two-step subsample-based estimator is also established. Extensive numerical experiments and a real-world example are provided to assess our subsampling strategy. Supplemental materials for this article are available online.

KEYWORDS:

• Asymptotic normality
• Empirical process
• L-optimality criterion
• Massive data
• Survival analysis

Acknowledgments

The authors would like to thank the Editor, the Associate Editor and two reviewers for their constructive and insightful comments that greatly improved the manuscript. We also thank Aliasghar Tarkhan for providing some helpful comments on the usage of R package “bigSurvSGD.”

Additional information

Funding

The work of Wang was supported by National Science Foundation (NSF), USA grant CCF-2105571. The work of Sun was supported in part by the National Natural Science Foundation of China (grant no. 12171463).