Learning to Forecast: The Probabilistic Time Series Forecasting Challenge

Abstract

We report on a course project in which students submit weekly probabilistic forecasts of two weather variables and one financial variable. This real-time format allows students to engage in practical forecasting, which requires a diverse set of skills in data science and applied statistics. We describe the context and aims of the course, and discuss design parameters like the selection of target variables, the forecast submission process, the evaluation of forecast performance, and the feedback provided to students. Furthermore, we describe empirical properties of students’ probabilistic forecasts, as well as some lessons learned on our part.

Keywords:

• Model evaluation and selection
• Prediction
• Quantile modeling
• Statistics education
• Time series analysis

Supplementary Materials

The supplementary materials contain additional empirical analyses.

Data Availability Statement

The repository containing students’ forecasts and all codes used to run the challenge is available at URL1. Codes to reproduce the tables and figures from this article are available at URL2. This repository also contains a simplified version of the interactive visualization dashboard created for the challenge, which can be used as a starting point for similar efforts.

Disclosure Statement

There are no competing interests to declare.

Acknowledgments

We thank our students for investing considerable time and effort into this novel course format. Furthermore, we acknowledge a student award generously provided by the International Institute of Forecasters (URL15). Marco Wurth and Benedikt Schulz provided excellent assistance with the weather data. Finally, we thank two anonymous reviewers for helpful comments on an earlier version of the article.

Additional information

Funding

Sebastian Lerch gratefully acknowledges support by the Vector Stiftung through the Young Investigator Group “Artificial Intelligence for Probabilistic Weather Forecasting.” Johannes Bracher acknowledges support by the Helmholtz Foundation via the SIMCARD Data Science Pilot Project.