3D reconstruction of partial foot scans using different state of the art neural network approaches

Figures & data

Figure 1. Operating principle of Shape Inversion GAN for shape completion (Zhang et al., 2021).

Figure 2. Operating principle of PoinTr for shape completion (Yu et al., 2021).

Figure 3. Operating principle of VRCNet for shape completion (L. Pan et al., 2021).

Figure 4. Operating principle of SnowflakeNet for shape completion (Xiang et al., 2021).

Figure 5. Operating principle of PointAttN for shape completion (J. Wang et al., 2022).

Figure 6. Baseline feet created from all feet of the training dataset.

Figure 7. Example point cloud from the training dataset.

Table 2. Quantitative results for all networks.

Metric		Baseline model	Shape Inversion	PoinTr	VRCNet	SnowflakeNet	PointAttN
Chamfer Distance	↓	0.0561	0.0336	0.0242	0.0370	0.0169	0.0352
Earth Mover’s Distance	↓	100.16	46.502	44.189	138.22	34.759	88.793
Hausdorff Distance	↓	0.2554	0.0657	0.0627	0.1388	0.0284	0.1163
F-Score	↑	0.2610	0.3360	0.4735	0.3615	0.6768	0.4693

Figure 8. Qualitative results of three point cloud completions from the different networks. Each row shows a different foot from the corpus.e dataset from two different views. The first column (ground truth (GT)) shows the best possible result of the reconstruction. The second column (input) displays what the network gets as a starting point for the reconstruction. The third column shows the baseline model, which represents an average foot. The following columns show the reconstructions of the different networks from the respective input.

Figure 9. Results of all metrics for each of the networks tested.

Table 1. Hyperparameters for Shape Inversion (Zhang et al., 2021), PoinTr (Yu et al., 2021), VRCNet (L. Pan et al., 2021), SnowflakeNet (Xiang et al., 2021) and PointAttN (J. Wang et al., 2022).

	Shape Inversion	PoinTr	VRCNet	SnowflakeNet	PointAttN
Learning rate	0.0001	0.0005	0.0001	0.001	0.0001
Epochs	2000	300	300	400	300
Batch size	2	4	13	1	8
Loss function	CD	CD L1	CD	CD L1	CD
Optimizer	Adam	Adam	Adam	Adam	Adam

Zhang, J., Chen, X., Cai, Z., Pan, L., Zhao, H., Yi, S., Yeo, C. K., Dai, B., & Loy, C. C. (2021, April). Unsupervised 3D shape completion through GAN inversion. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 1768–1777).

 Google Scholar

Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., & Zhou, J. (2021, August). PoinTr: Diverse point cloud completion with geometry-aware transformers. In Proceedings of the IEEE International Conference on Computer Vision (pp. 12478–12487).

 Google Scholar

Pan, L., Chen, X., Cai, Z., Zhang, J., Zhao, H., Yi, S., & Liu, Z. (2021, April). Variational relational point completion network. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 8520–8529).

 Google Scholar

Xiang, P., Wen, X., Liu, Y. S., Cao, Y. P., Wan, P., Zheng, W., & Han, Z. (2021, August). SnowflakeNet: Point cloud completion by snowflake point deconvolution with skip-transformer. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5479–5489).

 Google Scholar

Wang, J., Cui, Y., Guo, D., Li, J., Liu, Q., & Shen, C. (2022, March). PointAttN: You only need attention for point cloud completion. https://arxiv.org/abs/2203.08485v1

 Google Scholar