Advanced search
Publication Cover
International Journal of Systems Science Volume 55, 2024 - Issue 8
Submit an article Journal homepage
78
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

A model-free deep integral policy iteration structure for robust control of uncertain systems

Ding Wanga Faculty of Information Technology, Beijing University of Technology, Beijing, People's Republic of China;b Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, People's Republic of China;c Beijing Institute of Artificial Intelligence, Beijing University of Technology, Beijing, People's Republic of China;d Beijing Laboratory of Smart Environmental Protection, Beijing University of Technology, Beijing, People's Republic of ChinaCorrespondence[email protected]
View further author information
,
Ao Liua Faculty of Information Technology, Beijing University of Technology, Beijing, People's Republic of China;b Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, People's Republic of China;c Beijing Institute of Artificial Intelligence, Beijing University of Technology, Beijing, People's Republic of China;d Beijing Laboratory of Smart Environmental Protection, Beijing University of Technology, Beijing, People's Republic of ChinaView further author information
&
Junfei Qiaoa Faculty of Information Technology, Beijing University of Technology, Beijing, People's Republic of China;b Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, People's Republic of China;c Beijing Institute of Artificial Intelligence, Beijing University of Technology, Beijing, People's Republic of China;d Beijing Laboratory of Smart Environmental Protection, Beijing University of Technology, Beijing, People's Republic of ChinaView further author information
Pages 1571-1583 | Received 25 Sep 2023, Accepted 28 Jan 2024, Published online: 08 Feb 2024

References

  • Abu-Khalaf, M., & Lewis, F. L. (2005). Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica, 41(5), 779–791. https://doi.org/10.1016/j.automatica.2004.11.034
  • Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & W. E. Dixon (2013). A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica, 49(1), 82–92. https://doi.org/10.1016/j.automatica.2012.09.019
  • Duan, J., Li, J., Ge, Q., Li, S. E., Bujarbaruah, M., Ma, F., & Zhang, D. (2023). Relaxed actor-critic with convergence guarantees for continuous-time optimal control of nonlinear systems. IEEE Transactions on Intelligent Vehicles, 8(5), 3299–3311. https://doi.org/10.1109/TIV.2023.3255264
  • Guo, L., & Zhao, H. (2023). Online adaptive optimal control algorithm based on synchronous integral reinforcement learning with explorations. Neurocomputing, 520, 250–261. https://doi.org/10.1016/j.neucom.2022.11.055
  • Ha, M., Wang, D., & Liu, D. (2022). Discounted iterative adaptive critic designs with novel stability analysis for tracking control. IEEE/CAA Journal of Automatica Sinica, 9(7), 1262–1272. https://doi.org/10.1109/JAS.2022.105692
  • Jha, S. K., Roy, S. B., & Bhasin, S. (2019). Initial excitation-based iterative algorithm for approximate optimal control of completely unknown LTI systems. IEEE Transactions on Automatic Control, 64(12), 5230–5237. https://doi.org/10.1109/TAC.9
  • Jiang, H., Zhang, H., Luo, Y., & Han, J. (2019). Neural-network-based robust control schemes for nonlinear multiplayer systems with uncertainties via adaptive dynamic programming. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(3), 579–588. https://doi.org/10.1109/TSMC.6221021
  • Jiang, Y., & Jiang, Z. P. (2014). Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 25(5), 882–893. https://doi.org/10.1109/TNNLS.5962385
  • Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2018). Optimal and autonomous control using reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2042–2062. https://doi.org/10.1109/TNNLS.2017.2773458
  • Lee, J. Y., Park, J. B., & Choi, Y. H. (2014). On integral generalized policy iteration for continuous-time linear quadratic regulations. Automatica, 50(2), 475–489. https://doi.org/10.1016/j.automatica.2013.12.009
  • Liu, D., Li, C., Li, H., Wang, D., & Ma, H. (2015). Neural-network-based decentralized control of continuous-time nonlinear interconnected systems with unknown dynamics. Neurocomputing, 165, 90–98. https://doi.org/10.1016/j.neucom.2014.07.082
  • Modares, H., & Lewis, F. L. (2014). Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica, 50(7), 1780–1792. https://doi.org/10.1016/j.automatica.2014.05.011
  • Pang, B., Bian, T., & Jiang, Z. P. (2022). Robust policy iteration for continuous-time linear quadratic regulation. IEEE Transactions on Automatic Control, 67(1), 504–511. https://doi.org/10.1109/TAC.2021.3085510
  • Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2018). Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv preprint arXiv:1801.01236.
  • Vamvoudakis, K. G., & Lewis, F. L. (2010). Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 46(5), 878–888. https://doi.org/10.1016/j.automatica.2010.02.018
  • Vrabie, D., & Lewis, F. (2009). Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks, 22(3), 237–246. https://doi.org/10.1016/j.neunet.2009.03.008
  • Wallace, B. A., & Si, J. (in press). Continuous-time reinforcement learning control: A review of theoretical results, insights on performance, and needs for new designs. IEEE Transactions on Neural Networks and Learning Systems, 1–21. https://doi.org/10.1109/tnnls.2023.3245980
  • Wang, D., Cheng, L., & Yan, J. (2022). Self-learning robust control synthesis and trajectory tracking of uncertain dynamics. IEEE Transactions on Cybernetics, 52(1), 278–286. https://doi.org/10.1109/TCYB.2020.2979694
  • Wang, D., Gao, N., Liu, D., Li, J., & Lewis, F. (2024). Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications. IEEE/CAA Journal of Automatica Sinica, 11(1), 18–36. https://doi.org/10.1109/jas.2023.123843
  • Wang, D., He, H., & Liu, D. (2017). Adaptive critic nonlinear robust control: A survey. IEEE Transactions on Cybernetics, 47(10), 3429–3451. https://doi.org/10.1109/TCYB.2017.2712188
  • Wang, D., Hu, L., Zhao, M., & Qiao, J. (2023). Dual event-triggered constrained control through adaptive critic for discrete-time zero-sum games. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 53(3), 1584–1595. https://doi.org/10.1109/TSMC.2022.3201671
  • Wang, D., Li, C., Liu, D., & Mu, C. (2016). Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties. Information Sciences, 366, 121–133. https://doi.org/10.1016/j.ins.2016.05.034
  • Wang, D., Li, X., Zhao, M., & Qiao, J. (2024). Adaptive critic control design with knowledge transfer for wastewater treatment applications. IEEE Transactions on Industrial Informatics, 20(2), 1488–1497. https://doi.org/10.1109/tii.2023.3278875
  • Wang, D., & Mu, C. (2018). Adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a spring–mass–damper system. IEEE Transactions on Industrial Electronics, 65(1), 654–663. https://doi.org/10.1109/TIE.41
  • Wang, D., Ren, J., Ha, M., & Qiao, J. (2023). System stability of learning-based linear optimal control with general discounted value iteration. IEEE Transactions on Neural Networks and Learning Systems, 24(9), 6504–6514. https://doi.org/10.1109/tnnls.2021.3137524
  • Wang, D., Wang, J., Zhao, M., Xin, P., & Qiao, J. (2023). Adaptive multi-step evaluation design with stability guarantee for discrete-time optimal learning control. IEEE/CAA Journal of Automatica Sinica, 10(9), 1797–1809. https://doi.org/10.1109/JAS.2023.123684
  • Wang, W., Chen, X., Fu, H., & Wu, M. (2019). Data-driven adaptive dynamic programming for partially observable nonzero-sum games via Q-learning method. International Journal of Systems Science, 50(7), 1338–1352. https://doi.org/10.1080/00207721.2019.1599463
  • Wei, Q., Li, H., Yang, X., & He, H. (2021). Continuous-time distributed policy iteration for multicontroller nonlinear systems. IEEE Transactions on Cybernetics, 51(5), 2372–2383. https://doi.org/10.1109/TCYB.2020.2979614
  • Wu, Q., Zhao, B., & Liu, D. (2020). Adaptive dynamic programming-based decentralised control for large-scale nonlinear systems subject to mismatched interconnections with unknown time-delay. International Journal of Systems Science, 51(15), 2883–2898. https://doi.org/10.1080/00207721.2020.1803439
  • Xiao, G., Zhang, H., Zhang, K., & Wen, Y. (2018). Value iteration based integral reinforcement learning approach for H∞ controller design of continuous-time nonlinear systems. Neurocomputing, 285, 51–59. https://doi.org/10.1016/j.neucom.2018.01.029
  • Yang, J., Chen, W., & Li, S. (2023). Non-linear disturbance observer-based robust control for systems with mismatched disturbances/uncertainties. IET Control Theory and Applications, 5(18), 2053–2062. https://doi.org/10.1049/iet-cta.2010.0616
  • Yang, Y., Fan, X., Sun, B., Xu, C., Zuo, S., & Yue, D. (2022). Event-triggered adaptive approximately optimal tracking control of a class of non-affine SISO nonlinear systems via output feedback. International Journal of Systems Science, 53(2), 223–239. https://doi.org/10.1080/00207721.2021.1947410
  • Yang, Y., Gao, W., Modares, H., & Xu, C. Z. (2022). Robust actor–critic learning for continuous-time nonlinear systems with unmodeled dynamics. IEEE Transactions on Fuzzy Systems, 30(6), 2101–2112. https://doi.org/10.1109/TFUZZ.2021.3075501
  • Zhao, M., Wang, D., Qiao, J., Ha, M., & Ren, J. (2023). Advanced value iteration for discrete-time intelligent critic control: A survey. Artificial Intelligence Review, 56, 12315–12346. https://doi.org/10.1007/s10462-023-10497-1

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Order Reprints Request Corporate Permissions

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

Request Academic Permissions

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.