Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Abstract

This paper attempts to study the risk-sensitive discounted discrete-time Markov decision processes in Borel spaces, in which the reward functions are allowed to be unbounded from above and from below. We find mild conditions imposed on the primitive data of the decision processes, which not only ensure the existence of a solution to the optimality equation (OE in short), but also are the generalization of the bounded reward case. Furthermore, using the OE and a novel technique, we prove the existence of an optimal policy out of the class of randomized history-dependent policies. Finally, we illustrate our results with an inventory system.

Keywords:

• Markov decision process
• unbounded reward functions
• risk-sensitive discounted optimality
• optimality equation
• optimal policy

Acknowledgements

The author is indebted to the anonymous referees for many valuable comments and suggestions that have improved the presentation.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Research supported by the National Key Research and Development Program of China [grant number 2022YFA1004600] and National Natural Science Foundation of China (NSFC) [grant number 12301170].