Multi-frame super-resolution of remote sensing images using attention-based GAN models

Wang P., Sertel E.

Knowledge-Based Systems, vol.266, 2023 (SCI-Expanded) identifier

  • Publication Type: Article / Article
  • Volume: 266
  • Publication Date: 2023
  • Doi Number: 10.1016/j.knosys.2023.110387
  • Journal Name: Knowledge-Based Systems
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Applied Science & Technology Source, Computer & Applied Sciences, INSPEC, Library and Information Science Abstracts, Library, Information Science & Technology Abstracts (LISTA)
  • Keywords: Attention mechanism, GAN, Multi-frame, Satellite images, Super-resolution (SR)
  • Istanbul Technical University Affiliated: Yes

Abstract

Multi-frame super-resolution (MFSR) of remote sensing (RS) imageries becomes a critical research topic with the launch of new satellites having video capturing capability and the advancement of artificial intelligence techniques. In this study, an attention-based Generative Adversarial Network (GAN) algorithm is proposed for the multi-frame remote sensing image super-resolution (MRSISR). Firstly, we introduced an attention module to the generator and designed a space-based net that worked on every single frame for better temporal information extraction. Secondly, we proposed a novel attention module for better spatial and spectral information extraction. Thirdly, we applied an attention-based discriminator for the discriminative ability improvement of the discriminator. We implemented several experiments with the state-of-the-art models and the proposed approach using SpaceNet7 and Jilin-1 datasets. We quantitatively and qualitatively compared the results of different multi-frame super-resolution models.