UAV image matching from handcrafted to deep local features

ABSTRACT

Local feature matching between images is a challenging task, particularly when there are significant appearance variations, such as extreme viewpoint changes. In this work, we present LoFTRS, a deep learning-based image matching framework that integrates semantic constraints into the matching process. Our key insight is that a local feature matcher with deep layers can capture more human-intuitive and simpler-to-match features. In addition to image segmentation module, we also propose a detector-free Transformer module. It uses vector-based attention to model relevance among all features and achieves efficient and effective long-range context aggregation. Transformer module applies a relative position encoding to explicitly disclose relative distance information, further improving the representation of features. We evaluate the performance of LoFTRS comparing to various popular handcrafted and deep learning-based methods. We investigate the relationship between matching quality and the performance of subsequent processing steps, such as the accuracy and completeness of the model generated by SfM. The experimental results show that the proposed LoFTRS achieves equal or better image matching performance in terms of matching score, average track length, RMSE, and the number of 3D points.

KEYWORDS:

• Image processing feature matching
• handcrafted methods
• deep learning
• UAV
• SfM
• Transformer

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Due to the nature of the research, due to commercial supporting data is not available.