Research ArticleOpen AccessGoogle Scholar indexed
Asymptotic Evaluations of the Stability Index for a Markov Control Process with the Expected Total Discounted Reward Criterion
Departamento de Ciencias del Tecnológico de Monterrey, Campus Monterrey (ITESM-CM), Ciudad de Monterrey, N.L., México
- 1 Departamento de Ciencias del Tecnológico de Monterrey, Campus Monterrey (ITESM-CM), Ciudad de Monterrey, N.L., México
American Journal of Operations Research·Volume 11 (2021)·Pages 62–85·Published 13 January 2021·DOI10.4236/ajor.2021.111004
Copy link · social · email
Abstract
In this work, for a control consumption-investment process with the discounted reward optimization criteria, a numerical estimate of the stability index is made. Using explicit formulas for the optimal stationary policies and for the value functions, the stability index is explicitly calculated and through statistical techniques its asymptotic behavior is investigated (using numerical experiments) when the discount coefficient approaches 1. The results obtained define the conditions under which an approximate optimal stationary policy can be used to control the original process.
KeywordsControl Consumption-Investment ProcessDiscrete-Time Markov Control ProcessExpected Total Discounted RewardProbabilistic MetricsStability Index Estimation
- Dynkin, E.B. and Yushkevich, A.A. (1979) Controlled Markov Processes. Springer-Verlag, New York.
- Hernandez-Lerma, O. (1989) Adaptive Markov Control Process. Vol. 79, Springer-Verlang, New York. https://doi.org/10.1007/978-1-4419-8714-3
- Gordienko, E.I. (1992) An Estimate of the Stability of Optimal Control of Certain Stochastic and Deterministic Systems. Journal of Soviet Mathematics, 59, 891-899. https://doi.org/10.1007/BF01099115
- Gordienko, E.I. and Salem, F.S. (1998) Robustness Inequalities for Markov Control Processes with Unbounded Cost. Systems & Control Letters, 33, 125-130. https://doi.org/10.1016/S0167-6911(97)00077-7
- Gordienko, E.I. and Yushkevich, A.A. (2003) Stability Estimates in the Problem of Average Optimal Switching of a Markov Chain. Mathematical Methods of Operations Research, 57, 345-365. https://doi.org/10.1007/s001860200258
- Gordienko, E.I., Lemus-Rodriguez, E. and Montes-de-Oca, R. (2008) Discounted Cost Optimality Problem: Stability with Respect to Weak Metrics. Mathematical Methods of Operations Research, 68, 77-96. https://doi.org/10.1007/s00186-007-0171-z
- Gordienko, E., Martínez, J. and Ruiz de Chávez, J. (2015) Stability Estimation of Transient Markov Decision Processes. In: Mena, R.H., Pardo, J.C., Rivero, V. and Bravo, G.U., Eds., XI Symposium on Probability and Stochastic Processes, Mexico, 18-22 November 2013, 157-176. https://doi.org/10.1007/978-3-319-13984-5_8
- Martínez-Sánchez, J.E. (2020) Stability Estimation for Markov Control Processes with Discounted Cost. Applied Mathematics, 11, 491-509. https://doi.org/10.4236/am.2020.116036
- Gordienko, E.I. and Salem-Silva, F. (2000). Estimates of Stability of Markov Control Processes with Unbounded Costs. Kybernetika, 36, 195-210.
- Montes-de-Oca, R. and Salem-Silva, F. (2005) Estimates for Perturbations of Average Markov Decision Process with a Minimal State and Upper Bounded by Stochastically Ordered Markov Chains. Kybernetika, 41,757-772.
- Martinez, J. and Zaitzeva, E. (2015) Note on Stability Estimation in Average Markov Control Processes. Kybernetika, 51, 629-638. http://doi.org/10.14736/kyb-2015-4-0629
- Rachev, S.T. (1991) Probability Metrics and the Stability of Stochastic Models. Wiley, Chichester.
- Hernandez-Lerma, O. and Lasserre, J. (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York. https://doi.org/10.1007/978-1-4612-0729-0