A Two-Stage Convolutional Neural Network for Joint Demosaicking and Super-Resolution

Kan Chang, Hengxin Li, Yufei Tan, Pak Lun Kevin Ding, Baoxin Li

Research output: Contribution to journalArticlepeer-review

Abstract

As two practical and important image processing tasks, color demosaicking (CDM) and super-resolution (SR) have been studied for decades. However, most literature studies these two tasks independently, ignoring the potential benefits of a joint solution. In this paper, aiming at efficient and effective joint demosaicking and super-resolution (JDSR), a well-designed two-stage convolutional neural network (CNN) architecture is proposed. For the first stage, by making use of the sampling-pattern information, a pattern-aware feature extraction (PFE) module extracts features directly from the Bayer-sampled low-resolution (LR) image, while keeping the resolution of the extracted features the same as the input. For the second stage, a dual-branch feature refinement (DFR) module effectively decomposes the features into two components with different spatial frequencies, on which different learning strategies are applied. On each branch of the DFR module, the feature refinement unit, namely, densely-connected dual-path enhancement blocks (DDEB), establishes a sophisticated nonlinear mapping from the LR space to the high-resolution (HR) space. To achieve strong representational power, two paths of transformations and the channel attention mechanism are adopted in DDEB. Extensive experiments demonstrate that the proposed method is superior to the sequential combination of state-of-the-art (SOTA) CDM and SR methods. Moreover, with much smaller model size, our approach also surpasses other SOTA JDSR methods.

Original languageEnglish (US)
JournalIEEE Transactions on Circuits and Systems for Video Technology
DOIs
StateAccepted/In press - 2021
Externally publishedYes

Keywords

  • Color Demosaicking
  • Convolutional Neural Network
  • Convolutional neural networks
  • Feature extraction
  • Image color analysis
  • Image reconstruction
  • Image Restoration
  • Interpolation
  • Super-Resolution
  • Superresolution
  • Task analysis

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A Two-Stage Convolutional Neural Network for Joint Demosaicking and Super-Resolution'. Together they form a unique fingerprint.

Cite this