I consider a system whose deterioration follows a discrete-time and discrete-st ate Markov chain with an absorbing state. When the system is put into practice, I may select operation (wait), imperfect repair, or replacement at each d iscrete - time point. The true state of the system is not known when it is operated. Instead, the system is monitored after operation and some incomplete information concerned with the deterioration is obtained for decision making. Since there are multiple imperfect repairs, I can select one option from them when the imperfect repair is preferable to operation and replacement. To express this situation, I propose a POMDP model and theoretically investigate the structure of an optimal maintenance policy minimizing a total expected discounted cost for an unbounded horizon. Then two stochastic orders are used for the analysis of our problem.
Osaki, S. (2001) Stochastic Models in Reliability and Maintenance. Springer, Berlin. https://doi.org/10.1007/978-3-540-24808-8
Cho, D.I. and Parlar, M. (1991) A Survey of Maintenance Models for Multi-Unit Systems. European Journal of Operational Research, 51, 1-23. https://doi.org/10.1016/0377-2217(91)90141-H
Dekker, R. (1996) Applications of Maintenance Optimization Models: A Review and Analysis. Reliability Engineering and System Safety, 51, 229-240. https://doi.org/10.1016/0951-8320(95)00076-3
Wang, H. (2002) A Survey of Maintenance Policies of Deteriorating Systems European. Journal of Operational Research, 139, 469-489. https://doi.org/10.1016/S0377-2217(01)00197-7
Jardine, A., Lin, D. and Banjevic, D. (2006) A Review on Machinery Diagnostics and Prognostics Implementing Condition-Based Maintenance. Mechanical Systems and Signal Processing, 20, 1483-1510. https://doi.org/10.1016/j.ymssp.2005.09.012
Olde Keizer, M.C.A., Flapper, S.D.P. and Teunter, R.H. (2017) Condition-Based Maintenance Policies for Systems with Multiple Dependent Components: A Review. European Journal of Operational Research, 261, 405-420. https://doi.org/10.1016/j.ejor.2017.02.044
Alaswad, S. and Xiang, Y. (2017) A Review on Condition-Based Maintenance Optimization Models for Stochastically Deteriorating System. Reliability Engineering and System Safety, 157, 54-63. https://doi.org/10.1016/j.ress.2016.08.009
Derman, C. (1963) On Optimal Replacement Rules When Changes of States Are Markovian. In: Bellman, R., Ed., Mathematical Optimization Techniques, University of California Press, Berkley, 201-210. https://doi.org/10.1525/9780520319875-011
Douer, N. and Yechiali, U. (1994) Optimal Repair and Replacement in Markovian Systems. Communications in Statistics. Stochastic Models, 10, 253-270. https://doi.org/10.1080/15326349408807295
Hopp, W. and Wu, A. (1990) Machine Maintenance with Multiple Maintenance Actions. IIE Transactions, 22, 226-233. https://doi.org/10.1080/07408179008964177
Kurt, M. and Kharoufeh, J.P. (2010) Monotone Optimal Replacement Policies for a Markovian Deteriorating System in Controllable Environment. Operations Research Letters, 38, 273-279. https://doi.org/10.1016/j.orl.2010.03.001
Kurt, M. and Kharoufeh, J.P. (2010) Optimally Maintaining a Markovian Deteriorating System with Limited Imperfect Repairs. European Journal of Operational Research, 205, 368-380. https://doi.org/10.1016/j.ejor.2010.01.009
Tamura, N. (2011) An Optimal Inspection Policy for a Markovian Deteriorating System with Stochastically Decreasing Lifetime. International Journal of Reliability Quality and Operations Management, 2, 13-28.
Lam, C.T. and Yeh, R.H. (1994) Optimal Maintenance-Policies for Deteriorating Systems under Various Maintenance Strategies. IEEE Transactions on Reliability, 43, 423-430. https://doi.org/10.1109/24.326439
Lam, C.T. and Yeh, R.H. (1994) Optimal Replacement Policies for Multistate Deteriorating Systems. Naval Research Logistics, 41, 303-315. https://doi.org/10.1002/1520-6750(199404)41:3 3.0.CO;2-2
Lam, C.T. and Yeh, R.H. (1994) Comparison of Sequential and Continuous Inspection Strategies for Deteriorating Systems. Advances in Applied Probability, 26, 423-435. https://doi.org/10.2307/1427444
Yeh, R.H. (1996) Optimal Inspection and Replacement Policies for Multi-State Deteriorating Systems. European Journal of Operational Research, 96, 248-259. https://doi.org/10.1016/S0377-2217(96)00077-X
Yeh, R.H. (1997) State-Age-Dependent Maintenance Policies for Deteriorating Systems with Erlang Sojourn Time Distributions. Reliability Engineering and System Safety, 58, 55-60. https://doi.org/10.1016/S0951-8320(97)00049-5
Ross, S.M. (1971) Quality Control under Markovian Deterioration. Management Science, 17, 587-596. https://doi.org/10.1287/mnsc.17.9.587
Ohnishi, M., Kawai, H. and Mine, H. (1986) An Optimal Inspection and Replacement Policy under Incomplete State Information. European Journal of Operational Research, 27, 117-128. https://doi.org/10.1016/S0377-2217(86)80014-5
Grosfeld-Nir, A. (1996) A Two-State Partially Observable Markov Decision Process with Uniformly Distributed Observation. Operations Research, 44, 458-463. https://doi.org/10.1287/opre.44.3.458
Grosfeld-Nir, A. (2007) Control Limits for Two-State Partially Observable Markov Decision Processes. European Journal of Operational Research, 182, 300-304. https://doi.org/10.1016/j.ejor.2006.08.033
Chen, M., Fan, H., Hu, C. and Zhou, D. (2014) Maintaining Partially Observed Systems with Imperfect Observation and Resource Constraint. IEEE Transactions on Reliability, 63, 881-890. https://doi.org/10.1109/TR.2014.2337792
Srinivasan, R. and Parlikad, A.K. (2014) Semi-Markov Decision Process with Partial Information for Maintenance Decisions. IEEE Transactions on Reliability, 63, 891-898. https://doi.org/10.1109/TR.2014.2338811
Oosterom, C., Peng, H. and Houtm, G. (2017) Maintenance Optimization for a Markovian Deteriorating System with Population Heterogeneity. IISE Transactions, 49, 96-109. https://doi.org/10.1080/0740817X.2016.1205239
Kivanc, I., Ozgur-Unluakin, D. and Bilgic, T. (2022) Maintenance Policy Analysis of the Regenerative Air Heater System Using Factored POMDPs. Reliability Engineering and System Safety, 219, Article ID: 108195. https://doi.org/10.1016/j.ress.2021.108195
Ohnishi, M., Morioka, T. and Ibaraki, T. (1994) Optimal Minimal-Repair and Replacement Problem of Discrete-Time Markovian Deterioration System under Incomplete State Information. Computers and Industrial Engineering, 27, 409-412. https://doi.org/10.1016/0360-8352(94)90321-2
Pham, H. and Wang, H. (1996) Imperfect Maintenance. European Journal of Operational Research, 94, 425-438. https://doi.org/10.1016/S0377-2217(96)00099-9
Ivy, J.S. and Pollock, S.M. (2005) Marginally Monotonic Maintenance Policies for a Multi-State Deteriorating Machine with Probabilistic Monitoring, and Silent Failures. IEEE Transactions on Reliability, 54, 489-497. https://doi.org/10.1109/TR.2005.853443
Tamura, N., Yuge, T. and Yanagi, S. (2011) Monotone Properties of Optimal Maintenance Policy for Two-State Partially Observable Markov Decision Process Model with Multiple Observations. International Journal of Operations Research, 7, 23-34.
Fan, H., Xu, Z. and Chen, S.W. (2015) Optimally Maintaining a Multi-State System with Limited Imperfect Preventive Repairs. International Journal of Systems Science, 46, 1729-1740. https://doi.org/10.1080/00207721.2013.828799
Kijima, M. (1997) Markov Processes for Stochastic Modeling. Chapman & Hall, New York. https://doi.org/10.1007/978-1-4899-3132-0
Shaked, M. and Shanthikumar, J.G. (2007) Stochastic Orders. Springer, New York. https://doi.org/10.1007/978-0-387-34675-5
Monahan, G. (1982) A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. Management Science, 28, 1-111. https://doi.org/10.1287/mnsc.28.1.1
Guo, C. and Liang, Z. (2022) A Predictive Markov Decision Process for Optimizing Inspection and Maintenance Strategies of Partially Observable Multi-State Systems. Reliability Engineering and System Safety, 226, Article ID: 108683. https://doi.org/10.1016/j.ress.2022.108683