A new degradation model and an improved SRGAN for multi-image super-resolution reconstruction

ABSTRACT

In order to solve the problems existing in multi-image super-resolution reconstruction methods, such as the difficulty of acquiring and processing multiple low-resolution images, the inability to make full use of the complementary information between different images, and the loss of details, this paper proposes a new degradation model and an improved SRGAN for multi-image super-resolution reconstruction. Firstly, a new degradation model considering both masked autoencoding and downsampling (DMMD) is designed, which can simulate the complex degradation environment in real life well and reduce the difficulty of obtaining and processing multiple low-resolution images. Then, we design a weight setting strategy for image fusion to make full use of the complementary information between different low-resolution images. To enhance the network's attention and propagation of high-frequency effective information in the feature map, the convolution block attention module (CBAM) is introduced into the generator network of SRGAN model, and a SRGAN combined with CBAM (SRGANCBAM) is designed. Finally, the fused low-resolution image is input into SRGANCBAM for reconstruction to obtain its corresponding high-resolution image. The experimental results show that our DMMD can solve the problem that it is difficult to acquire and process multiple low-resolution images. The complementary information between different images is fully utilized by the reasonable weight setting strategy. On four public datasets, the PSNR value of our SRGANCBAM can be 0.532, 0.207, 0.357 and 0.537 higher than the baseline model, respectively. Compared with the state-of-the-art methods, our SRGANCBAM achieves higher values of evaluation metrics and reconstructs clearer and more realistic images.

KEYWORDS:

• Multi-image super-resolution
• masked autoencoding
• degradation model
• image fusion
• CBAM

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Funding

This work was partly supported by the Natural Science Basis Research Plan in Shaanxi Province of China [grant number 2023-JC-YB-517], the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems Beihang University [grant number VRLAB2023B08], and the high-level talent introduction project of Shaanxi Technical College of Finance & Economics [grant number 2022KY01].

Notes on contributors

Hongan Li

Hongan Li received the M.S. degree in 2009 and the Ph.D. degree in 2014 in computer science and technology from Northwest University, Shaanxi, China. From 2019 to 2020, he worked as a domestic visiting scholar at Tsinghua University. From 2017 to 2024, he was an associate professor at the College of Computer Science and Technology, Xi'an University of Science and Technology. His research interests include computer graphics and computer-aided geometric design, computer animation, virtual reality, and image processing.

Lizhi Cheng

Lizhi Cheng was born in 2000 in Weinan, Shaanxi Province, China. She received her bachelor's degree in software engineering from the College of Computer Science and Technology, Xi 'an University of Science and Technology in 2022. She is currently studying for a master's degree in the College of Computer Science and Technology, Xi'an University of Science and Technology. Her research interests include the use of image processing and computer vision.

Jun Liu

Jun Liu graduated from the School of Computer Science and Technology, Chengdu University of Technology, for the degree of Bachelor in 1995. He received his M.S. degree in the School of Computer Science and Technology, Northwest University in 2009. Since 2011 he worked toward his Ph.D. degree at Northwest University and received his Ph.D. degree in computer science from Northwest University in 2018. He entered the National-Local Joint Engineering Research Center of Cultural Heritage Digitization in Northwest University, as a researcher from 2016 to 2019. He is a member of China Computer Federation. His current research interests include cultural heritage digitization, pattern recognition and machine learning.