Benefiting from multitask learning to improve single image super-resolution


Rad M. S. , Bozorgtabar B., Musat C., Marti U., Basler M., Ekenel H. K. , ...More

NEUROCOMPUTING, vol.398, pp.304-313, 2020 (Journal Indexed in SCI) identifier identifier

  • Publication Type: Article / Article
  • Volume: 398
  • Publication Date: 2020
  • Doi Number: 10.1016/j.neucom.2019.07.107
  • Title of Journal : NEUROCOMPUTING
  • Page Numbers: pp.304-313

Abstract

Despite significant progress toward super resolving more realistic images by deeper convolutional neural networks (CNNs), reconstructing fine and natural textures still remains a challenging problem. Recent works on single image super resolution (SISR) are mostly based on optimizing pixel and content wise similarity between recovered and high-resolution (HR) images and do not benefit from recognizability of semantic classes. In this paper, we introduce a novel approach using categorical information to tackle the SISR problem; we present an encoder architecture able to extract and use semantic information to super-resolve a given image by using multitask learning, simultaneously for image super-resolution and semantic segmentation. To explore categorical information during training, the proposed decoder only employs one shared deep network for two task-specific output layers. At run-time only layers resulting HR image are used and no segmentation label is required. Extensive perceptual experiments and a user study on images randomly selected from COCO-Stuff dataset demonstrate the effectiveness of our proposed method and it outperforms the state-of-the-art methods. (C) 2019 Elsevier B.V. All rights reserved.