Distributed Censored Quantile Regression

Abstract

This article discusses an extension of censored quantile regression to a distributed setting. With the growing availability of massive datasets, it is oftentimes an arduous task to analyze all the data with limited computational facilities efficiently. Our proposed method, which attempts to overcome this challenge, is comprised of two key steps, namely: (i) estimation of both Kaplan-Meier estimator and model coefficients in a parallel computing environment; (ii) aggregation of coefficient estimations from individual machines. We study the upper limit of the order of the number of machines for this computing environment, which, if fulfilled, guarantees that the proposed estimator converges at a comparable rate to that of the oracle estimator. In addition, we also provide two further modifications for distributed systems including (i) a communication-facilitated adaptation in the sense of Chen, Liu, and Zhang and (ii) a nonparametric counterpart along the direction of Kong and Xia for censored quantile regression. Numerical experiments are conducted to compare the proposed and the existing estimators. The promising results demonstrate the computation efficiency of the proposed methods. Finally, for practical concerns, a cross-validation procedure is also developed which can better select the hyperparameters for the proposed methodologies. Supplementary materials for this article are available online.

KEYWORD:

• Distributed computing
• Lifetime and survival analysis
• Quantile estimation regression

Supplemental Materials

Supplementary material includes details of the proofs of technical results.

Acknowledgments

The authors would like to express their tremendous gratitude to the editor, the associate editor and the anonymous referees for their valuable comments and suggestions which improve the presentation and substantiates the discussion of the revised manuscript.

Notes

1 https://www.mn.uio.no/math/english/people/aca/borgan/abg-data.html

2 https://www.kaggle.com/c/kkbox-churn-prediction-challenge/overview

Additional information

Funding

The first author would like to acknowledge the financial support of Hong Kong Research grants Council RGC-14301618, RGC-14301920, and RGC-14307221.