An approximate dynamic programming approach for the maintenance optimisation of networked critical infrastructures

References

• Arruda, E. F., Fragoso, M. D., & do Val, J. B. R. (2011). Approximate dynamic programming via direct search in the space of value function approximations. European Journal of Operational Research, 211(2), 343–351. https://doi.org/10.1016/j.ejor.2010.11.019
   Web of Science ®Google Scholar
• Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2/3), 235–256. https://doi.org/10.1023/A:1013689704352
   Web of Science ®Google Scholar
• Bertsekas, D. P. (2005). Dynamic programming and optimal control (3rd ed., Vol. I). Athena Scientific.
   Google Scholar
• Bertsekas, D. P. (2009). Neuro-dynamic programmingneuro-dynamic programming. In C. A. Floudas & P. M. Pardalos (Eds.), Encyclopedia of optimization (p. 2555–2560). Springer US. https://doi.org/10.1007/978-0-387-74759-0_440
   Google Scholar
• Bobbio, A., Ferraris, C., Terruggia, R. (2006, November). New challenges in network reliability analysis (Report No.TR-INF-2006-11-04-UNIPMN No. TR-INF-2006-11-04-UNIPMN). Alessandria, Italy: Computer Science Institute - Università del Piemonte Orientale “A. Avogadro”. http://www.di.unipmn.it/TechnicalReports/TR-INF-2006-11-04-UNIPMN.pdf
   Google Scholar
• Boland, N., Kalinowski, T., & Kaur, S. (2015). Scheduling network maintenance jobs with release dates and deadlines to maximize total flow over time: Bounds and solution strategies. Computers & Operations Research, 64, 113–129. https://doi.org/10.1016/j.cor.2015.05.011
   Web of Science ®Google Scholar
• Boland, N., Kalinowski, T., Waterer, H., & Zheng, L. (2014). Scheduling arc maintenance jobs in a network to maximize total flow over time. Discrete Applied Mathematics, 163, 34–52. https://doi.org/10.1016/j.dam.2012.05.027
   Web of Science ®Google Scholar
• Cassady, C. R., Pohl, E. A., & Murdock, W. P. (2001). Selective maintenance modeling for industrial systems. Journal of Quality in Maintenance Engineering, 7(2), 104–117. https://doi.org/10.1108/13552510110397412
   Google Scholar
• Conrad, S. H., LeClaire, R. J., Reilly, G. P. O., & Uzunalioglu, H. (2006). Critical national infrastructure reliability modeling and analysis. Bell Labs Technical Journal, 11(3), 57–71. https://doi.org/10.1002/bltj.20178
   Web of Science ®Google Scholar
• Dahlquist, G., & Bjorck, A. (2008). Interpolation and approximation. In D. Germund & B. Åke (Eds.), Numerical methods in scientific computing (Vol. I, pp. 351–519) Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9780898717785.ch4
   Google Scholar
• Dantzig, G. B., & Fulkerson, D. R. (1956). On the max-flow min-cut theorem of networks. In H. W. Kuhn & A. Tucker (Eds.), Linear inequalities and related systems (pp. 215–221). Princeton University Press.
   Google Scholar
• Durango-Cohen, P. L., & Madanat, S. M. (2008). Optimization of inspection and maintenance decisions for infrastructure facilities under performance model uncertainty: A quasi-bayes approach. Transportation Research Part A: Policy and Practice, 42(8), 1074–1085. https://doi.org/10.1016/j.tra.2008.03.004
   Web of Science ®Google Scholar
• Goel, A., & Meisel, F. (2013). Workforce routing and scheduling for electricity network maintenance with downtime minimization. European Journal of Operational Research, 231(1), 210–228. https://doi.org/10.1016/j.ejor.2013.05.021
   Web of Science ®Google Scholar
• Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., Inc.
   Google Scholar
• González, A. D., Chapman, A., Dueñas‐Osorio, L., Mesbahi, M., & D’Souza, R. M. (2017). Efficient infrastructure restoration strategies using the recovery operator. Computer-Aided Civil and Infrastructure Engineering, 32(12), 991–1006. https://doi.org/10.1111/mice.12314
   Web of Science ®Google Scholar
• Hu, X., Daganzo, C., & Madanat, S. (2015). A reliability-based optimization scheme for maintenance management in large-scale bridge networks. Transportation Research Part C: Emerging Technologies, 55, 166–178. https://doi.org/10.1016/j.trc.2015.01.008
   Web of Science ®Google Scholar
• Jia, Q. S. (2010). A structural property of optimal policies for multi-component maintenance problems. IEEE Transactions on Automation Science & Engineering, 7(3), 677–680. https://doi.org/10.1109/TASE.2009.2036375
   Web of Science ®Google Scholar
• Jia, Q. S. (2011). On state aggregation to approximate complex value functions in large-scale markov decision processes. IEEE Transactions on Automatic Control, 56(2), 333–344. https://doi.org/10.1109/TAC.2010.2052697
   Web of Science ®Google Scholar
• Jianhua, W., Yangji, S., & Haijun, F. (1989). A method for the vulnerability evaluation of a directed flow network with both edge and vertex failures [Paper presentation]. IEEE International Symposium on Circuits and Systems (pp. 2000–2003). https://doi.org/10.1109/ISCAS.1989.100764
   Google Scholar
• Khatab, A., & Aghezzaf, E. H. (2016). Selective maintenance optimization when quality of imperfect maintenance actions are stochastic. Reliability Engineering & System Safety, 150, 182–189. https://doi.org/10.1016/j.ress.2016.01.026
   Web of Science ®Google Scholar
• Kijima, M. (1989). Some results for repairable systems with general repair. Journal of Applied Probability, 26(1), 89–102. https://doi.org/10.2307/3214319
   Web of Science ®Google Scholar
• Kijima, M., Morimura, H., & Suzuki, Y. (1988). Periodical replacement problem without assuming minimal repair. European Journal of Operational Research, 37(2), 194–203. https://doi.org/10.1016/0377-2217(88)90329-3
   Web of Science ®Google Scholar
• Lee, J. H., & Lee, J. M. (2006). Approximate dynamic programming based approach to process control and scheduling. Computers & Chemical Engineering, 30(10–12), 1603–1618. https://doi.org/10.1016/j.compchemeng.2006.05.043
   Web of Science ®Google Scholar
• Lie, C. H., & Chun, Y. H. (1986). An algorithm for preventive maintenance policy. IEEE Transactions on Reliability, 35(1), 71–75. https://doi.org/10.1109/Tr.1986.4335352.
   Web of Science ®Google Scholar
• Lin, P. M., Leon, B. J., & Huang, T. C. (1976). A new algorithm for symbolic system reliability analysis. IEEE Transactions on Reliability, R-25(1), 2–15. https://doi.org/10.1109/TR.1976.5214942
   Web of Science ®Google Scholar
• Liscouski, B., Elliot, W. (2004). Final report on the august 14, 2003 blackout in the United States and Canada: Causes and recommendations (Report No.AR-1165). U.S.-Canada Power System Outage Task Force. https://www3.epa.gov/region1/npdes/merrimackstation/pdfs/ar/AR-1165.pdf
   Google Scholar
• Liu, Y., & Huang, H.-Z. (2010). Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Transactions on Reliability, 59(2), 356–367. https://doi.org/10.1109/tr.2010.2046798
   Web of Science ®Google Scholar
• Maillart, L. M., Cassady, C. R., Rainwater, C., & Schneider, K. (2009). Selective maintenance decision-making over extended planning horizons. IEEE Transactions on Reliability, 58(3), 462–469. https://doi.org/10.1109/TR.2009.2026689
   Web of Science ®Google Scholar
• Malik, M. (1979). Reliable preventive maintenance scheduling. A I I E Transactions, 11(3), 221–228. https://doi.org/10.1080/05695557908974463
   Web of Science ®Google Scholar
• Nourelfath, M., Fitouhi, M. C., & Machani, M. (2010). An integrated model for production and preventive maintenance planning in multi-state systems. IEEE Transactions on Reliability, 59(3), 496–506. https://doi.org/10.1109/TR.2010.2056412
   Web of Science ®Google Scholar
• Novoa, C., & Storer, R. (2009). An approximate dynamic programming approach for the vehicle routing problem with stochastic demands. European Journal of Operational Research, 196(2), 509–515. https://doi.org/10.1016/j.ejor.2008.03.023
   Web of Science ®Google Scholar
• Pandey, M., Zuo, M. J., & Moghaddass, R. (2012). Selective maintenance for binary systems using age-based imperfect repair model [Paper presentation]. International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, 2012 (pp. 385–389). https://doi.org/10.1109/ICQR2MSE.2012.6246259
   Google Scholar
• Pandey, M., Zuo, M. J., & Moghaddass, R. (2016). Selective maintenance scheduling over a finite planning horizon. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 230(2), 162–177. https://doi.org/10.1177/1748006X15598914
   Web of Science ®Google Scholar
• Pandey, M., Zuo, M. J., Moghaddass, R., & Tiwari, M. K. (2013). Selective maintenance for binary systems under imperfect repair. Reliability Engineering & System Safety, 113, 42–51. https://doi.org/10.1016/j.ress.2012.12.009
   Web of Science ®Google Scholar
• Patriksson, M., Strömberg, A.-B., & Wojciechowski, A. (2015a). The stochastic opportunistic replacement problem, part ii: A two-stage solution approach. Annals of Operations Research, 224(1), 51–75. https://doi.org/10.1007/s10479-012-1134-1
   Web of Science ®Google Scholar
• Patriksson, M., Strömberg, A.-B., & Wojciechowski, A. (2015b). The stochastic opportunistic replacement problem, part i: Models incorporating individual component lives. Annals of Operations Research, 224(1), 25–50. https://doi.org/10.1007/s10479-012-1131-4
   Web of Science ®Google Scholar
• Powell, W. B. (2011). Approximate dynamic programming: Solving the curses of dimensionality. John Wiley & Sons.
   Google Scholar
• Rajagopalan, R., & Cassady, C. R. (2006). An improved selective maintenance solution approach. Journal of Quality in Maintenance Engineering, 12(2), 172–185. https://doi.org/10.1108/13552510610667183
   Google Scholar
• Rice, W., Cassady, C., Nachlas, J. (1998, May). Optimal maintenance plans under limited maintenance time. In Proceedings of the Seventh Industrial Engineering Research Conference (pp. 1–3).
   Google Scholar
• Roohnavazfar, M., Manerba, D., De Martin, J. C., & Tadei, R. (2019). Optimal paths in multi-stage stochastic decision networks. Operations Research Perspectives, 6, 100124. https://doi.org/10.1016/j.orp.2019.100124
   Web of Science ®Google Scholar
• Schneider, K., & Cassady, C. R. (2015). Evaluation and comparison of alternative fleet-level selective maintenance models. Reliability Engineering & System Safety, 134, 178–187. https://doi.org/10.1016/j.ress.2014.10.017
   Web of Science ®Google Scholar
• Sun, T., Zhao, Q., & Luh, P. B. (2007). Incremental value iteration for time-aggregated Markov-decision processes. IEEE Transactions on Automatic Control, 52(11), 2177–2182. https://doi.org/10.1109/TAC.2007.908359
   Web of Science ®Google Scholar
• Tadei, R., Perboli, G., & Manerba, D. (2020). The multi-stage dynamic stochastic decision process with unknown distribution of the random utilities. Optimization Letters, 14(5), 1207–1218. https://doi.org/10.1007/s11590-019-01412-1
   Web of Science ®Google Scholar
• Tawarmalani, M., & Li, Y. (2011). Multi-period maintenance scheduling of tree networks with minimum flow disruption. Naval Research Logistics (NRL), 58(5), 507–530. https://doi.org/10.1002/nav.20455
   Web of Science ®Google Scholar
• Topaloglu, H. (2005). An approximate dynamic programming approach for a product distribution problem. IIE Transactions, 37(8), 697–710. https://doi.org/10.1080/07408170590918083
   Web of Science ®Google Scholar
• Ulmer, M. W., Goodson, J. C., Mattfeld, D. C., & Hennig, M. (2018). Offline–online approximate dynamic programming for dynamic vehicle routing with stochastic requests. Transportation Science, 52(1), 20–37. https://doi.org/10.1287/trsc.2017.0767
   Web of Science ®Google Scholar
• Usberti, F. L., Lyra, C., Cavellucci, C., & González, J. F. V. (2015). Hierarchical multiple criteria optimization of maintenance activities on power distribution networks. Annals of Operations Research, 224(1), 171–192. https://doi.org/10.1007/s10479-012-1182-6
   Web of Science ®Google Scholar
• Wong, W. C., & Lee, J. H. (2011). Postdecision-state-based approximate dynamic programming for robust predictive control of constrained stochastic processes. Industrial & Engineering Chemistry Research, 50(3), 1389–1399. https://doi.org/10.1021/ie1007882
   Web of Science ®Google Scholar
• Zhang, C., Ramirez-Marquez, J. E., & Rocco, C. M. (2011). A holistic method for reliability performance assessment and critical components detection in complex networks. IIE Transactions, 43(9), 661–675. https://doi.org/10.1080/0740817X.2010.546387
   Web of Science ®Google Scholar
• Zio, E., & Piccinelli, R. (2010). Randomized flow model and centrality measure for electrical power transmission network analysis. Reliability Engineering & System Safety, 95(4), 379–385. https://doi.org/10.1016/j.ress.2009.11.008.s
   Web of Science ®Google Scholar