Estimation of log-gripping position using instance segmentation for autonomous log loading

References

• Azadeh K, De Koster R, Roy D. 2019. Robotized and automated warehouse systems: review and recent developments. Transp Sci. 53(4):917–945. doi: 10.1287/trsc.2018.0873.
   Web of Science ®Google Scholar
• Bauwens S, Bartholomeus H, Calders K, Lejeune P. 2016. Forest inventory with terrestrial lidar: a comparison of static and hand-held mobile laser scanning. Forests. 7(12):127. doi: 10.3390/f7060127.
   Web of Science ®Google Scholar
• Bewley A, Ge Z, Ott L, Ramos F, Upcroft B. 2016. Simple online and realtime tracking. Proceeding of 2016 IEEE International Conference on Image Processing; Sep 25-28; Phoenix, AZ. p. 3464–3468.
   Google Scholar
• Bolya D, Zhou C, Xiao F, Lee YJ. 2019. YOLACT: real-time instance segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision; Oct 27- Nov 2; Seoul, Korea (South). p. 9157–9166.
   Google Scholar
• Bolya D, Zhou C, Xiao F, Lee YJ. 2022. YOLACT++: better real-time instance segmentation. IEEE Trans Pattern Anal Mach Intell. 44(2):1108–1121. doi: 10.1109/TPAMI.2020.3014297.
   PubMed Web of Science ®Google Scholar
• Cheng B, Misra I, Schwing AG, Kirillov A, Girdhar R. 2022. Masked-attention mask transformer for universal image segmentation. Proceeding of IEEE/CVF Conference on Computer Vision and Pattern Recognition; Jun 18-24; New Orleans, LA. p. 1290–1299.
   Google Scholar
• Couceiro MS, Portugal D, Ferreira JF, Rocha RP. 2019. SEMFIRE: towards a new generation of forestry maintenance multi-robot systems. IEEE/SICE International Symposium System Integration SII. 2019; Jan 14-16; Paris, France: IEEE/SICE. p. 270–276.
   Google Scholar
• Cubuk ED, Zoph B, Shlens J, Le QV. 2020. RandAugment: practical automated data augmentation with a reduced search space. Proceeding of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops; Jun 14-19; Seattle, WA. p. 3008–3017.
   Google Scholar
• Dadhich S, Bodin U, Andersson U. 2016. Key challenges in automation of earth-moving machines. Autom Constr. 68:212–222. doi: 10.1016/j.autcon.2016.05.009.
   Web of Science ®Google Scholar
• da Silva DQ, dos Santos FN, Filipe V, Sousa AJ, Oliveira PM. 2022. Edge AI-based tree trunk detection for forestry monitoring robotics. Robotics. 11(6):136. doi: 10.3390/robotics11060136.
   Web of Science ®Google Scholar
• Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A. 2010. The pascal visual object classes (VOC) challenge. Int J Comput Vis. 88(2):303–338. doi: 10.1007/s11263-009-0275-4.
   Web of Science ®Google Scholar
• Fantoni G, Santochi M, Dini G, Tracht K, Scholz-Reiter B, Fleischer J, Kristoffer Lien T, Seliger G, Reinhart G, Franke J, et al. 2014. Grasping devices and methods in automated production processes. CIRP Ann. 63(2):679–701. doi: 10.1016/J.CIRP.2014.05.006.
   Web of Science ®Google Scholar
• Fortin J-M, Gamache O, Grondin V, Pomerleau FF, Giguere P, Giguère P. 2022. Instance segmentation for autonomous log grasping in forestry operations. Proceeding of IEEE International Conference on Intelligent Robots and Systems; Oct 23-27; Kyoto, Japan. p. 6064–6071.
   Google Scholar
• Geiger C, Beiser S, Geimer M. 2021. Automated driving on a skid road with a forwarder in a ctl logging process. Proceedings of The Joint 43rd Annual Meeting of Council on Forest Engineering (COFE) & the 53rd International Symposium on Forest Mechanization (FORMEC); Sep 27-30; Corvallis, OR, USA. p. 135.
   Google Scholar
• Geiger C, Maier N, Kalinke F, Geimer M. 2020. Assistance system for an automated log-quality and assortment estimation based on data-driven approaches using hydraulic signals of forestry machines. Proceeding of 12th International Fluid Power Conference; Oct 12-14; Dresden, Germany. p. 83–92
   Google Scholar
• Geiger C, Weißenböck M, Geimer M. 2021. Assistance system for an automatic loading process. proceedings of the joint 43rd annual meeting of council on forest engineering (COFE) & the 53rd International Symposium on Forest Mechanization (FORMEC) (); Sep 27-30; Corvallis, OR, USA. p. 5–7.
   Google Scholar
• Ghiasi G, Cui Y, Srinivas A, Qian R, Lin TY, Cubuk ED, Le QV, Zoph B. 2020. Simple copy-paste is a strong data augmentation method for instance segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Jun 20-25; Nashville, TN. p. 2918–2928.
   Google Scholar
• Gholamy A, Kreinovich V, Kosheleva O. 2018. Why 70/30 or 80/20 relation between training and testing sets: a pedagogical explanation; [accessed 2022 Nov 14]. https://scholarworks.utep.edu/cs_techrep/1209.
   Google Scholar
• Girshick R. 2015. Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision; Dec 7-13; Santiago, Chile. p. 1440–1448.
   Google Scholar
• Girshick R, Donahue J, Darrell T, Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Jun 23-28; Columbus, OH. p. 580–587.
   Google Scholar
• Grondin V, Fortin JM, Pomerleau F, Giguère P. 2023. Tree detection and diameter estimation based on deep learning. For An Int J For Res. 96(2):264–276. doi: 10.1093/forestry/cpac043.
   Google Scholar
• Grondin V, Pomerleau F, Giguère P. 2022. Training deep learning algorithms on synthetic forest images for tree detection; [accessed 2023 Mar 17]. https://arxiv.org/abs/2210.04104v1.
   Google Scholar
• He K, Gkioxari G, Dollar P, Girshick R, Dollár P, Girshick R. 2020. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. 42(2):386–397. doi: 10.1109/TPAMI.2018.2844175.
   PubMed Web of Science ®Google Scholar
• Hellström T, Johansson T, Ringdahl O. 2006. Development of an autonomous forest machine for path tracking. In: Corke P, and Sukkariah S, editors Field and service robotics. Heidelberg, Berlin: Springer; p. 603–614.
   Google Scholar
• Hellström T, Nordfjell T, Ringdahl O. 2009. Autonomous forest vehicles: historic, envisioned, and state-of-the-art. Int J For Eng. 20(1):31–38. doi: 10.1080/14942119.2009.10702573.
   Google Scholar
• Huang Z, Huang L, Gong Y, Huang C, Wang X. 2019. Mask scoring R-CNN. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Jun 15-20; Long Beach, CA. p. 6402–6411.
   Google Scholar
• Hunčaga M, Chudá J, Tomaštík J, Slámová M, Koreň M, Chudý F. 2020. The comparison of stem curve accuracy determined from point clouds acquired by different terrestrial remote sensing methods. Remote Sens. 12(17):2739. doi: 10.3390/rs12172739.
   Google Scholar
• Hyyppä E, Yu X, Kaartinen H, Hakala T, Kukko A, Vastaranta M, Hyyppä J. 2020. Comparison of backpack, handheld, under-canopy UAV, and above-canopy UAV laser scanning for field reference data collection in boreal forests. Remote Sens. 12(20):3327. doi: 10.3390/rs12203327.
   Google Scholar
• Iglhaut J, Cabo C, Puliti S, Piermattei L, O’Connor J, Rosette J. 2019. Structure from motion photogrammetry in forestry: a review. Curr For Reports. 5(3):155–168. doi: 10.1007/s40725-019-00094-3.
   Google Scholar
• Jia W, Zhang Y, Lian J, Zheng Y, Zhao D, Li C. 2020. Apple harvesting robot under information technology: a review. Int J Adv Robot Syst. 17(3):172988142092531. doi: 10.1177/1729881420925310.
   Web of Science ®Google Scholar
• Kalman RE. 1960. A new approach to linear filtering and prediction problems. J Basic Eng. 82(1):35–45. doi: 10.1115/1.3662552.
   Google Scholar
• Kootstra G, Wang X, Blok PM, Hemming J, van Henten E. 2021. Selective harvesting robotics: current research, trends, and future directions. Curr Robot Reports. 2(1):95–104. doi: 10.1007/s43154-020-00034-1.
   Google Scholar
• Kuhn HW. 1955. The Hungarian method for the assignment problem. Nav Res Logist Q. 2(1–2):83–97. doi: 10.1002/nav.3800020109.
   Google Scholar
• Lindroos O, Ringdahl O, La Hera P, Hohnloser P, Hellström TH. 2015. Estimating the position of the harvester head–a key step towards the precision forestry of the future? Croat J For Eng. 36(2):147–164.
   Web of Science ®Google Scholar
• Lulea university of technology. 2021. First in the world with autonomous forwarding. [accessed 2023 Apr 6]. https://www.ltu.se/research/subjects/maskinkonstruktion/Nyheter-och-aktuellt/Forst-i-varlden-med-autonom-skotning-1.212384?l=en.
   Google Scholar
• Merzlyakov A, MacEnski S. 2021. A comparison of modern general-purpose visual slam approaches. Proceeding of IEEE International Conference on Intelligent Robots and Systems; Sep 28-30; Prague, Czech Republic. p. 9190–9197.
   Google Scholar
• Mowshowitz A, Tominaga A, Hayashi E. 2018. Robot navigation in forest management. J Robot Mechatronics. 30(2):223–230. doi: 10.20965/jrm.2018.p0223.
   Web of Science ®Google Scholar
• Redmon J, Divvala S, Girshick R, Farhadi A. 2016. You only look once: unified, real-time object detection. Proceeding of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Jun 26-Jul 1; Las Vegas, NV. p. 779–788.
   Google Scholar
• Ren S, He K, Girshick R, Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 39(6):1137–1149. doi: 10.1109/TPAMI.2016.2577031.
   PubMed Web of Science ®Google Scholar
• Ringdahl O, Lindroos O, Hellström T, Bergström D, Athanassiadis D, Nordfjell T. 2011. Path tracking in forest terrain by an autonomous forwarder. Scand J For Res. 26(4):350–359. doi: 10.1080/02827581.2011.566889.
   Web of Science ®Google Scholar
• Sharafutdinov D, Griguletskii M, Kopanev P, Kurenkov M, Ferrer G, Burkov A, Gonnochenko A, Tsetserukou D. 2023. Comparison of modern open-source visual SLAM approaches. J Intell Robot Syst. 107(3):1–22. doi: 10.1007/s10846-023-01812-7.
   Web of Science ®Google Scholar
• Smolyanskiy N, Kamenev A, Smith J, Birchfield S. 2017. Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness. IEEE International Conference on Intelligent Robots and Systems. Sep 24-28; Vancouver, BC, Canada. p. 4241–4247.
   Google Scholar
• Stereolabs. 2023. ZED SDK. [accessed 2023 Jul 10]. https://www.stereolabs.com/developers/release/#downloads.
   Google Scholar
• Tai K, El-Sayed AR, Shahriari M, Biglarbegian M, Mahmud S. 2016. State of the art robotic grippers and applications. Robotics. 5(2):11. doi: 10.3390/robotics5020011.
   Web of Science ®Google Scholar
• Usui K. 2021. Log shape segmentation by deep learning for autonomous log loading. Proceeding of The Joint 43rd Annual Meeting of Council on Forest Engineering (COFE) & the 53rd International Symposium on Forest Mechanization (FORMEC); Sep 27-30; Corvallis, OR, USA. p. 270–275.
   Google Scholar
• Usui K. 2023. Estimation of grappling log position using a stereo camera for autonomous log-loading. J Japan For Eng Soc. 38(1):35–42. Japanese. doi: 10.18945/jjfes.38.35.
   Google Scholar
• Usui K, Mozuna M, Uemura T, Nakazawa M. 2019. Log detection based on deep convolutional neural networks. Proceeding of the 52nd International Symposium on Forest Mechanization (FORMEC); Oct 6-10; Sopron, Hungary; p. 469–474.
   Google Scholar
• Visser R, Obi OF. 2021. Automation and robotics in forest harvesting operations. Croat J For Eng. 42(1):13–24. doi: 10.5552/crojfe.2021.739.
   Web of Science ®Google Scholar
• Wallace L, Lucieer A, Malenovský Z, Turner D, Vopěnka P. 2016. Assessment of forest structure using two uav techniques: a comparison of airborne laser scanning and structure from motion (SfM) point clouds. Forests. 7(12):62. doi: 10.3390/f7030062.
   Web of Science ®Google Scholar
• Zhou H, Wang X, Au W, Kang H, Chen C. 2022. Intelligent robots for fruit harvesting: recent developments and future challenges. Precis Agric. 23(5):1856–1907. doi: 10.1007/s11119-022-09913-3.
   Web of Science ®Google Scholar