Advanced search
Publication Cover
Connection Science Volume 35, 2023 - Issue 1
Submit an article Journal homepage
1,065
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

Zhihan XueDepartment of Information & Communication Sciences, Faculty of Science & Technology, Sophia University, Tokyo, JapanCorrespondence[email protected]
&
Tad GonsalvesDepartment of Information & Communication Sciences, Faculty of Science & Technology, Sophia University, Tokyo, Japan
Article: 2183828 | Received 05 Jun 2022, Accepted 17 Feb 2023, Published online: 08 Mar 2023

References

  • Anderson, W. C., Carey, K., Sturzinger, E. M., & Lowrance, C. J. (2019). Autonomous Navigation via a deep Q network with one-hot image encoding. 2019 IEEE International Symposium on Measurement and Control in Robotics (ISMCR). https://doi.org/10.1109/ismcr47492.2019.8955697
  • Azar, A. T., Koubaa, A., Ali Mohamed, N., Ibrahim, H. A., Ibrahim, Z. F., Kazim, M., Ammar, A., Benjdira, B., Khamis, A. M., Hameed, I. A., & Casalino, G. (2021). Drone deep reinforcement learning: A review. Electronics, 10(9), 999. https://doi.org/10.3390/electronics10090999
  • Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. https://doi.org/10.1109/tpami.2013.50
  • Bonatti, R., Madaan, R., Vineet, V., Scherer, S., & Kapoor, A. (2020). Learning visuomotor policies for aerial navigation using cross-modal representations. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://doi.org/10.1109/iros45743.2020.9341049
  • Caltagirone, L., Bellone, M., Svensson, L., & Wahde, M. (2019). LIDAR–camera fusion for road detection using fully convolutional neural networks. Robotics and Autonomous Systems, 111, 125–131. https://doi.org/10.1016/j.robot.2018.11.002
  • Caselles-Dupré, H., Garcia-Ortiz, M., & Filliat, D. (2018, December 11). Continual state representation learning for reinforcement learning using Generative Replay. arXiv.org. Retrieved October 25, 2022, from https://arxiv.org/abs/1810.03880
  • Devo, A., Mezzetti, G., Costante, G., Fravolini, M. L., & Valigi, P. (2020). Towards generalization in target-driven visual navigation by using deep reinforcement learning. IEEE Transactions on Robotics, 36(5), 1546–1561. https://doi.org/10.1109/tro.2020.2994002
  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021, June 3). An image is worth 16(16 words: Transformers for image recognition at scale. arXiv.org. Retrieved June 3, 2022, from https://arxiv.org/abs/2010.11929
  • Elkholy, H. A., Azar, A. T., Shahin, A. S., Elsharkawy, O. I., & Ammar, H. H. (2020). Path planning of a self driving vehicle using artificial intelligence techniques and machine vision. Advances in Intelligent Systems and Computing, 1153, 532–542. https://doi.org/10.1007/978-3-030-44289-7_50
  • Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., & Kavukcuoglu, K. (2018, June 28). Impala: Scalable distributed deep-RL with importance weighted actor-learner architectures. arXiv.org. Retrieved June 3, 2022, from https://arxiv.org/abs/1802.01561
  • Finn, C., Tan, X. Y., Duan, Y., Darrell, T., Levine, S., & Abbeel, P. (2016). Deep spatial autoencoders for Visuomotor Learning. 2016 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra.2016.7487173
  • Fujimoto, S., van Hoof, H., & Meger, D. (2018, October 22). Addressing function approximation error in actor-critic methods. arXiv.org. Retrieved June 3, 2022, from https://arxiv.org/abs/1802.09477
  • Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., & Levine, S. (2019, January 29). Soft actor-critic algorithms and applications. arXiv.org. Retrieved June 3, 2022, from https://arxiv.org/abs/1812.05905v2
  • Hasselt, H. v., Guez, A., & Silver, D. (2016, February 1). Deep reinforcement learning with double Q-learning: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Guide Proceedings. Retrieved June 3, 2022, from https://dl.acm.org/doi/10.55553016100.3016191
  • He, D., Zou, Z., Chen, Y., Liu, B., Yao, X., & Shan, S. (2021). Obstacle detection of rail transit based on deep learning. Measurement, 176, 109241. https://doi.org/10.1016/j.measurement.2021.109241
  • Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017, April 17). MobileNets: Efficient convolutional neural networks for Mobile Vision Applications. arXiv.org. Retrieved January 23, 2023, from https://arxiv.org/abs/1704.04861
  • Kumar, V. R., Klingner, M., Yogamani, S., Milz, S., Fingscheidt, T., & Mader, P. (2021). SynDistNet: Self-supervised monocular fisheye camera distance estimation synergised with semantic segmentation for autonomous driving. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.1109/wacv48630.2021.00011
  • Lange, S., & Riedmiller, M. (2010). Deep auto-encoder neural networks in reinforcement learning. The 2010 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn.2010.5596468
  • Lee, H. Y., Ho, H. W., & Zhou, Y. (2020). Deep learning-based monocular obstacle avoidance for unmanned aerial vehicle navigation in tree plantations. Journal of Intelligent & Robotic Systems, 101, 1. https://doi.org/10.1007/s10846-020-01284-z
  • Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2019, July 5). Continuous control with deep reinforcement learning. arXiv.org. Retrieved June 3, 2022, fromhttps://arxiv.org/abs/1509.02971
  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. Computer Vision – ECCV, 2016, 21–37. https://doi.org/10.1007/978-3-319-46448-0_2
  • Lu, Y., Xue, Z., Xia, G.-S., & Zhang, L. (2018). A survey on vision-based UAV navigation. Geo-Spatial Information Science, 21(1), 21–32. https://doi.org/10.1080/10095020.2017.1420509
  • Ma, Z., Wang, C., Niu, Y., Wang, X., & Shen, L. (2018). A saliency-based reinforcement learning approach for a UAV to avoid flying obstacles. Robotics and Autonomous Systems, 100, 108–118. https://doi.org/10.1016/j.robot.2017.10.009
  • Nachum, O., Gu, S., Lee, H., & Levine, S. (2018, October 5). Data-efficient hierarchical reinforcement learning. arXiv.org. Retrieved January 22, 2023, from https://arxiv.org/abs/1805.08296v4
  • Ota, K., Oiki, T., Jha, D. K., Mariyama, T., & Nikovski, D. (2020, June 27). Can increasing input dimensionality improve deep reinforcement learning? arXiv.org. Retrieved August 16, 2022, from https://arxiv.org/abs/2003.01629
  • Padhy, R. P., Verma, S., Ahmad, S., Choudhury, S. K., & Sa, P. K. (2018). Deep neural network for autonomous UAV navigation in indoor corridor environments. Procedia Computer Science, 133, 643–650. https://doi.org/10.1016/j.procs.2018.07.099
  • Pham, D., & Le, T. (2020). Auto-encoding variational Bayes for inferring topics and visualization. Proceedings of the 28th International Conference on Computational Linguistics. https://doi.org/10.18653/v1/2020.coling-main.458
  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2016.91
  • Ruiz-del-Solar, J., Loncomilla, P., & Soto, N. (2018, March 28). A survey on deep learning methods for robot vision. arXiv.org. Retrieved March 2, 2023, from https://arxiv.org/abs/1803.10862
  • Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2018.00474
  • Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2017). Airsim: High-fidelity visual and physical simulation for autonomous vehicles. Field and Service Robotics, 5, 621–635. https://doi.org/10.1007/978-3-319-67361-5_40
  • Shin, S.-Y., Kang, Y.-W., & Kim, Y.-G. (2020). Reward-driven U-Net training for obstacle avoidance drone. Expert Systems with Applications, 143, 113064. https://doi.org/10.1016/j.eswa.2019.113064
  • Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529(7587), 484–489. https://doi.org/10.1038/nature16961
  • Singh, A., Kalaichelvi, V., & Karthikeyan, R. (2022). A survey on vision guided robotic systems with intelligent control strategies for autonomous tasks. Cogent Engineering, 9(1), https://doi.org/10.1080/23311916.2022.2050020
  • Sutton, R. S., McAllester, D., Singh, S., & Mansour, Y. (1999, November 1). Policy gradient methods for reinforcement learning with function approximation: Proceedings of the 12th International Conference on Neural Information Processing Systems. Guide Proceedings. Retrieved June 3, 2022, from https://dl.acm.org/doi/10.5555/3009657.3009806
  • Wang, Z., Schaul, T., Hessel, M., Hasselt, H. V., Lanctot, M., & Freitas, N. D. (2016, June 1). Dueling network architectures for Deep Reinforcement Learning: Proceedings of the 33rd International Conference on International Conference on Machine Learning - volume 48. Guide Proceedings. Retrieved June 3, 2022, from https://dl.acm.org/doi/10.55553045390.3045601
  • Wu, Y., Mansimov, E., Liao, S., Radford, A., & Schulman, J. (2017, August 18). OpenAI Baselines: ACKTR & A2c. OpenAI. Retrieved June 3, 2022, from https://openai.com/blog/baselines-acktr-a2c/
  • Xiao, X., Liu, B., Warnell, G., & Stone, P. (2022). Motion planning and control for mobile robot navigation using machine learning: A survey. Autonomous Robots, 46, 569–597. https://doi.org/10.1007/s10514-022-10039-8
  • Yue, P., Xin, J., Zhao, H., Liu, D., Shan, M., & Zhang, J. (2019). Experimental research on deep reinforcement learning in autonomous navigation of Mobile Robot. 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA). https://doi.org/10.1109/iciea.2019.8833968
  • Zhang, J., Hao, B., Chen, B., Li, C., Chen, H., & Sun, J. (2019). Hierarchical reinforcement learning for course recommendation in moocs. Proceedings of the AAAI Conference on Artificial Intelligence, 33((01|1)), 435–442. https://doi.org/10.1609/aaai.v33i01.3301435
  • Zhou, X., Bai, T., Gao, Y., & Han, Y. (2019). Vision-based robot navigation through combining unsupervised learning and hierarchical reinforcement learning. Sensors, 19(7), 1576. https://doi.org/10.3390/s19071576
  • Zhou, Y., & Ho, H. W. (2022). Online robot guidance and navigation in non-stationary environment with hybrid hierarchical reinforcement learning. Engineering Applications of Artificial Intelligence, 114, 105152. https://doi.org/10.1016/j.engappai.2022.105152
  • Zieliński, P., & Markowska-Kaczmar, U. (2021). 3D robotic navigation using a vision-based deep reinforcement learning model. Applied Soft Computing, 110, 107602. https://doi.org/10.1016/j.asoc.2021.107602

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.