Publication Cover
Canadian Journal of Remote Sensing
Journal canadien de télédétection
Volume 50, 2024 - Issue 1
544
Views
0
CrossRef citations to date
0
Altmetric
Research Article

3D Modeling of Façade Elements Using Multi-View Images from Mobile Scanning Systems

Modélisation 3D d‘éléments de façade à l‘aide d‘images multi-vues provenant de systèmes de numérisation mobiles

, ORCID Icon, &
Article: 2309895 | Received 06 Sep 2023, Accepted 21 Jan 2024, Published online: 21 Feb 2024

References

  • Chen, Y., Liu, X., Yao, M., Deng, S., Li, F., Cheng, L., and Li, M. 2019. “Repeated Structure Detection for 3D Reconstruction of Building Façade from Mobile Lidar Data.” Photogrammetric Engineering & Remote Sensing, Vol. 85 (No. 2):pp. 1–16. doi:10.14358/PERS.85.2.93.
  • Cheng, B., Chen, S., Fan, L., Li, Y., Cai, Y., and Liu, Z. 2023. “Windows and Doors Extraction from Point Cloud Data Combining Semantic Features and Material Characteristics.” Buildings, Vol. 13 (No. 2):pp. 507. doi:10.3390/buildings13020507.
  • Collins, T., Mesejo, P., and Bartoli, A. 2014. “An Analysis of Errors in Graph-Based Keypoint Matching and Proposed Solutions.” In Computer Vision – ECCV 2014, edited by David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, 138–153. Lecture Notes in Computer Science. Cham: Springer International Publishing. doi:10.1007/978-3-319-10584-0_10.
  • Esri. 2022. https://www.arcgis.com/home/item.html?id=8c0078cc7e314e31b20001d94daace5e
  • Gröger, G., and Plümer, L. 2012. “CityGML – Interoperable Semantic 3D City Models.” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 71 (No. July):pp. 12–33. doi:10.1016/j.isprsjprs.2012.04.004.
  • Han, Y., Choi, J., Byun, Y., and Kim, Y. 2014. “Parameter Optimization for the Extraction of Matching Points Between High-Resolution Multisensor Images in Urban Areas.” IEEE Transactions on Geoscience and Remote Sensing, Vol. 52 (No. 9):pp. 5612–5621. doi:10.1109/TGRS.2013.2291001.
  • Jung, J., Stachniss, C., Ju, S., and Heo, J. 2018. “Automated 3D Volumetric Reconstruction of Multiple-Room Building Interiors for as-Built BIM.” Advanced Engineering Informatics, Vol. 38 (No. October):pp. 811–825. (Elsevier: doi:10.1016/j.aei.2018.10.007.
  • Kamra, V., Kudeshia, P., Arabinaree, S., Chen, D., Akiyama, Y., and Peethambaran, J. 2023. “Lightweight Reconstruction of Urban Buildings: Data Structures, Algorithms, and Future Directions.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 16 pp. 902–917. Institute of Electrical and Electronics Engineers Inc.: doi:10.1109/JSTARS.2022.3232758.
  • Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., et al. 2023. “Segment Anything.” https://arxiv.org/abs/2304.02643v1
  • Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., et al. 2018. “The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale.” International Journal of Computer Vision, Vol. 128 (No. 7):pp. 1956–1981. doi:10.1007/s11263-020-01316-z.
  • Li, C.-K., Zhang, H.-X., Liu, J.-X., Zhang, Y.-Q., Zou, S.-C., and Fang, Y.-T. 2020. “Window Detection in Facades Using Heatmap Fusion.” Journal of Computer Science and Technology, Vol. 35 (No. 4):pp. 900–912. doi:10.1007/s11390-020-0253-4.
  • Li, Z., Zhang, L., Mathiopoulos, P.T., Liu, F., Zhang, L., Li, S., and Liu, H. 2017. “A Hierarchical Methodology for Urban Facade Parsing from TLS Point Clouds.” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 123 (No. January):pp. 75–93. doi:10.1016/j.isprsjprs.2016.11.008.
  • Liu, H., Li, W., and Zhu, J. 2022. “Translational Symmetry-Aware Facade Parsing for 3D Building Reconstruction.” IEEE Multimedia., Vol. 29 (No. 4):pp. 38–47. doi:10.1109/MMUL.2022.3195990.
  • Liu, H., Xu, Y., Zhang, J., Zhu, J., Li, Y., and Hoi, S.C.H. 2020. “DeepFacade: A Deep Learning Approach to Facade Parsing with Symmetric Loss.” IEEE Transactions on Multimedia, Vol. 22 (No. 12):pp. 3153–3165. (Institute of Electrical and Electronics Engineers Inc.: doi:10.1109/TMM.2020.2971431.
  • Lotte, R.G., Haala, N., Karpina, M., de Aragão, L.E.O., Cruz, e., and Shimabukuro, Y.E. 2018. “3D Façade Labeling over Complex Scenarios: A Case Study Using Convolutional Neural Network and Structure-From-Motion.” Remote Sensing, Vol. 10 (No. 9):pp. 1435. doi:10.3390/rs10091435.
  • Lu, Y., Wei, W., Li, P., Zhong, T., Nong, Y., and Shi, X. 2023. “A Deep Learning Method for Building Façade Parsing Utilizing Improved SOLOv2 Instance Segmentation.” Energy and Buildings, Vol. 295 (No. September):pp. 113275. doi:10.1016/j.enbuild.2023.113275.
  • Ma, W., Ma, W., and Xu, S. 2022. “Deep Facade Parsing with Occlusions.” KSII Transactions on Internet and Information Systems, Vol. 16 (No. 2):pp. 524–543. doi:10.3837/TIIS.2022.02.009.
  • Malihi, S., Zoej, M.J.V., Hahn, M., and Mokhtarzade, M. 2018. “Window Detection from UAS-Derived Photogrammetric Point Cloud Employing Density-Based Filtering and Perceptual Organization.” Remote Sensing, Vol. 10 (No. 8):pp. 1320. doi:10.3390/rs10081320.
  • Moghimi, A., Celik, T., and Mohammadzadeh, A. 2022. “Tensor-Based Keypoint Detection and Switching Regression Model for Relative Radiometric Normalization of Bitemporal Multispectral Images.” International Journal of Remote Sensing, Vol. 43 (No. 11):pp. 3927–3956. doi:10.1080/01431161.2022.2102951.
  • Neuhausen, M., and König, M. 2018. “Automatic Window Detection in Facade Images.” Automation in Construction, Vol. 96 (No. December):pp. 527–539. doi:10.1016/j.autcon.2018.10.007.
  • Nordmark, N., and Ayenew, M. 2021. “Window Detection In Facade Imagery: A Deep Learning Approach Using Mask R-CNN,” July. https://arxiv.org/abs/2107.10006v1.
  • Pantoja-Rosero, B.G., Achanta, R., Kozinski, M., Fua, P., Perez-Cruz, F., and Beyer, K. 2022. “Generating LOD3 Building Models from Structure-from-Motion and Semantic Segmentation.” Automation in Construction, Vol. 141 (No. September):pp. 104430. doi:10.1016/j.autcon.2022.104430.
  • Pexman, K., Lichti, D.D., and Dawson, P. 2021. “Automated Storey Separation and Door and Window Extraction for Building Models from Complete Laser Scans.” Remote Sensing, Vol. 13 (No. 17):pp. 3384. doi:10.3390/rs13173384.
  • Previtali, M., Díaz-Vilariño, L., and Scaioni, M. 2018. “Towards Automatic Reconstruction of Indoor Scenes From Incomplete Point Clouds: Door and Window Detection and Regularization.” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLII-4 pp. 507–514. doi:10.5194/isprs-archives-XLII-4-507-2018.
  • Pu, S., and Vosselman, G. 2009. “Knowledge Based Reconstruction of Building Models from Terrestrial Laser Scanning Data.” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 64 (No. 6):pp. 575–584. doi:10.1016/j.isprsjprs.2009.04.001.
  • Ren, S., He, K., Girshick, R., and Sun, J. 2015. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39 (No. 6):pp. 1137–1149. doi:10.1109/TPAMI.2016.2577031.
  • Salehi, A., and Mohammadzadeh, A. 2017. “Building Roof Reconstruction Based on Residue Anomaly Analysis and Shape Descriptors from Lidar and Optical Data.” Photogrammetric Engineering & Remote Sensing, Vol. 83 (No. 4):pp. 281–291. doi:10.14358/PERS.83.4.281.
  • Shepard, D. 1968. “A Two-Dimensional Interpolation Function for Irregularly-Spaced Data.” Paper presented at Proceedings of the 1968 23rd ACM National Conference On. ACM Press, 517–524. doi:10.1145/800186.810616.
  • Sun, Y., Malihi, S., Li, H., and Maboudi, M. 2022. “DeepWindows: Windows Instance Segmentation through an Improved Mask R-CNN Using Spatial Attention and Relation Modules.” ISPRS International Journal of Geo-Information, Vol. 11 (No. 3):pp. 162. doi:10.3390/ijgi11030162.
  • Teeravech, K., Nagai, M., Honda, K., and Dailey, M. 2014. “Discovering Repetitive Patterns in Facade Images Using a RANSAC-Style Algorithm.” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 92 (No. June):pp. 38–53. doi:10.1016/j.isprsjprs.2014.02.018.
  • Torr, P.H.S., and Zisserman, A. 2000. “MLESAC: A New Robust Estimator with Application to Estimating Image Geometry.” Computer Vision and Image Understanding, Vol. 78 (No. 1):pp. 138–156. doi:10.1006/cviu.1999.0832.
  • Tripodi, S., Duan, L., Poujade, V., Trastour, F., Bauchet, J. P., Laurore, L., and Tarabalka, Y. 2020. “Operational Pipeline for Large-Scale 3D Reconstruction of Buildings from Satellite Images.” Paper presented at International Geoscience and Remote Sensing Symposium (IGARSS), September. Institute of Electrical and Electronics Engineers Inc., 445–448. doi:10.1109/IGARSS39084.2020.9324213.
  • Wang, S., Kang, Q., She, R., Tay, W. P., Navarro, D. N., and Hartmannsgruber, A. 2022. “Building Facade Parsing R-CNN,” May. https://arxiv.org/abs/2205.05912v1.
  • Wang, W., Xu, Y., Ren, Y., and Wang, G. 2021. “Parsing of Urban Facades from 3D Point Clouds Based on a Novel Multi-View Domain.” Photogrammetric Engineering & Remote Sensing, Vol. 87 (No. 4):pp. 283–293. (American Society for Photogrammetry and Remote Sensing: doi:10.14358/PERS.87.4.283.
  • Wang, Y., Zhao, Y., and Petzold, L. 2023. “An Empirical Study on the Robustness of the Segment Anything Model (SAM),” May. https://arxiv.org/abs/2305.06422v2.
  • Wen, X., Xie, H., Liu, H., and Yan, L. 2019. “Accurate Reconstruction of the LoD3 Building Model by Integrating Multi-Source Point Clouds and Oblique Remote Sensing Imagery.” ISPRS International Journal of Geo-Information, Vol. 8 (No. 3):pp. 135. doi:10.3390/ijgi8030135.
  • Wolf, P. R., Dewitt, B. A., and Wilkinson, B. E. 2014. “Elements of Photogrammetry” (with Applications in GIS). 4th ed. New York: McGraw-Hill Education.
  • Wysocki, O., Hoegner, L., and Stilla, U. 2022. “Refinement of Semantic 3D Building Models by Reconstructing Underpasses from MLS Point Clouds.” International Journal of Applied Earth Observation and Geoinformation, Vol. 111 (No. July):pp. 102841. doi:10.1016/j.jag.2022.102841.
  • Xia, S., and Wang, R. 2019. “Façade Separation in Ground-Based LiDAR Point Clouds Based on Edges and Windows.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 12 (No. 3):pp. 1041–1052. doi:10.1109/JSTARS.2019.2897987.
  • Zhang, C., Puspitasari, F.D., Zheng, S., Li, C., Qiao, Y., Kang, T., Shan, X. 2023. “A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering.” arXiv doi:10.48550/arXiv.2306.06211.
  • Zhang, G., Pan, Y., and Zhang, L. 2022. “Deep Learning for Detecting Building Façade Elements from Images Considering Prior Knowledge.” Automation in Construction, Vol. 133 (No. January):pp. 104016. doi:10.1016/j.autcon.2021.104016.
  • Zheng, Y., Peter, M., Zhong, R., Elberink, S.O., and Zhou, Q. 2018. “Space Subdivision in Indoor Mobile Laser Scanning Point Clouds Based on Scanline Analysis.” Sensors (Basel, Switzerland), Vol. 18 (No. 6):pp. 1838. doi:10.3390/S18061838.
  • Zhuo, X., Tian, J., and Fraundorfer, F. 2023. “Cross Field-Based Segmentation and Learning-Based Vectorization for Rectangular Windows.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16: 431–448. doi:10.1109/JSTARS.2022.3218767.