Unintended look-ahead bias in out-of-sample forecasting

ABSTRACT

This article shows that out-of-sample tests are susceptible to look-ahead bias not only to multiple testing problem that is emphasized in the literature. A forecaster often constructs a well-performing model without trial and error but with an intuition that is derived from observed empirical patterns in the test sample. Such an intuition, however, is unavailable in the beginning of the test sample. Therefore, the reported forecasting performance in an out-of-sample test is possibly exaggerated, although a forecaster simply utilizes her expertise without any intended p-hacking or fishing. A stylized forecasting model with an example of stock market return predictability quantitatively demonstrates this unintended look-ahead bias in out-of-sample tests.

KEYWORDS:

• Forecasting
• out-of-sample test
• look-ahead bias
• model selection bias

JEL CLASSIFICATION:

• C10
• C44
• C53

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ http://www.hec.unil.ch/agoyal/.

² https://fred.stlouisfed.org/series/GDPC1.