A mean-variance optimization problem for discounted Markov decision processes

详细信息	查看全文 | 推荐本文 |

• 作者：Xianping Guo^a ; ¹ ; ^{mcsgxp@mail.sysu.edu.cn} ; Liuer Ye^c ; ¹ ; ^{yeliuer@hotmail.com} ; George Yin^b ; ² ; ^{gyin@math.wayne.edu}
• 关键词：Mean&ndash ; variance criterion ; Finite continuous-time MDPs ; Discounted reward ; Policy iteration algorithm ; Efficient frontier
• 刊名：European Journal of Operational Research
• 出版年：2012
• 期刊代码：58_03772217
• 类别：et
• 出版时间：16 July, 2012
• 卷：220
• 期：2
• 页码：423-429
• 文件大小：254 K

摘要

In this paper, we consider a mean-variance optimization problem for Markov decision processes (MDPs) over the set of (deterministic stationary) policies. Different from the usual formulation in MDPs, we aim to obtain the mean-variance optimal policy that minimizes the variance over a set of all policies with a given expected reward. For continuous-time MDPs with the discounted criterion and finite-state and action spaces, we prove that the mean-variance optimization problem can be transformed to an equivalent discounted optimization problem using the conditional expectation and Markov properties. Then, we show that a mean-variance optimal policy and the efficient frontier can be obtained by policy iteration methods with a finite number of iterations. We also address related issues such as a mutual fund theorem and illustrate our results with an example.