3D Modeling of Façade Elements Using Multi-View Images from Mobile Scanning Systems

Figures & data

Figure 1. Four levels of detail (LoDs) represent building models. As the level of detail increases, the geometric detail and semantic complexity also increase.

Figure 2. The windows are obstructed by trees and billboard.

Figure 3. Dataset compilation from single-sided facades (FC 1- FC 12).

Table 1. EOPs accuracies for two datasets, coordinates in meters, and angles in degree unit.

		EOPs accuracy
		${\sigma }_{X_L}(m)$	${\sigma }_{Y_L}(m)$	${\sigma }_{Z_L}(m)$	${\sigma }_{\omega }{}^{°}$	${\sigma }_{\phi }{}^{°}$	${\sigma }_{\kappa }{}^{°}$
Dataset	SfM	0.001	0.001	0.001	0.013	0.013	0.006
	MLS	0.01	0.01	0.02	0.013	0.013	0.004

Table 2. The specifications of data collection devices.

Device	Sensor	Specifications
Canon EOS 5D Mark III DSLR camera	Frame camera	Number of cameras: 1 Sensor width: 37.10 mm Sensor height: 24.73 mm Sensor: Full frame Data: RGB image, pixels Width: 5760 pixels Height: 3840 pixels Pixel size: 6.44 µm
Leica Pegasus: Two Ultimate Laser scanning device	Frame and Pavement cameras	Number of cameras: 4 Sensor: CMOS 12 MP (megapixels) Data: RGB image, pixels Width: 3008 pixels Height: 4096 pixels Pixel size: 3.45 µm
	Scanner	Number of scanners: 1 Type: Z + F PROFILER® 9012 Mode: single Data: 3D point cloud [X, Y, Z], m Point density: ∼ 12000 point/m^2

Figure 4. The overall flowchart of the proposed method.

Figure 5. The pseudo-code of the proposed method.

Figure 6. The overlaid bounding boxes (green) on single view façade images captured by DSLR cameras (a), and Leica MLS device (b).

Figure 7. a) A single view façade image captured Fredericton residential area b) detected windows and doors bounding boxes by Faster-RCNN c) refined 2D SAM segments.

Figure 8. a) A single view façade image captured Fredericton residential area b) projected point cloud overlaid on the image.

Figure 9. 2D-3D correspondence and buffering process. (a) projected point cloud on a single view façade image b) overlaid 2D border segment (red color) c) SfM point cloud d) buffered corresponding 3D point cloud, e) removed noisy window points.

Figure 10. The process to generate facade elements 3D locations. a) A single view façade image captured UNB campus b) overlaid a 2D polygon from SAM (red color) c) the projected 3D polygon overlaid on SfM point cloud d) zoomed in the projected 3D polygon.

Figure 11. Three samples of overlapping 3D polygons (window border candidates) across three image views. a) Overlapping 3D polygons for a single window b) 2D polygons in different view angles.

Figure 12. Overlaid 3D polygons on the single-sided façade point clouds (FC 1-12).

Figure 13. Detected incorrect objects (yellow dash line circles) in the two different image views.

Table 3. Mean and standard deviation of IoU values for the final 3D polygons of facade elements, and the number of façade elements and images used in the implementation.

Façade #	Number of images	Number of windows	Number of doors	IoU [0 - 1] (unitless)
				$\overline{\mathit{X}}$ $\pm$σ
FC 1	137	42	0	0.82 $\pm$ 0.08
FC 2	15	7	2	0.90 $\pm$ 0.02
FC 3	15	18	1	0.81 $\pm$ 0.14
FC 4	69	40	2	0.81 $\pm$ 0.16
FC 5	47	23	2	0.84 $\pm$ 0.17
FC 6	55	21	1	0.76 $\pm$ 0.21
FC 7	170	43	1	0.89 $\pm$ 0.04
FC 8	77	28	1	0.89 $\pm$ 0.15
FC 9	40	11	1	0.85 $\pm$ 0.20
FC 10	40	16	0	0.89 $\pm$ 0.05
FC 11	28	11	1	0.80 $\pm$ 0.33
FC 12	57	40	0	0.81 $\pm$ 0.14
Total	750	300	12	-
Average	-	-	-	0.84$\pm$0.16

The values of column 5 are dimensionless decimal numbers, indicating the ratio of overlap.