Comparison of deep learning and conventional demosaicing algorithms for mastcam images

Chiman Kwan, Bryan Chou, James Bell

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Bayer pattern filters have been used in many commercial digital cameras. In National Aeronautics and Space Administration’s (NASA) mast camera (Mastcam) imaging system, onboard the Mars Science Laboratory (MSL) rover Curiosity, a Bayer pattern filter is being used to capture the RGB (red, green, and blue) color of scenes on Mars. The Mastcam has two cameras: left and right. The right camera has three times better resolution than that of the left. It is well known that demosaicing introduces color and zipper artifacts. Here, we present a comparative study of demosaicing results using conventional and deep learning algorithms. Sixteen left and 15 right Mastcam images were used in our experiments. Due to a lack of ground truth images for Mastcam data from Mars, we compared the various algorithms using a blind image quality assessment model. It was observed that no one algorithm can work the best for all images. In particular, a deep learning-based algorithm worked the best for the right Mastcam images and a conventional algorithm achieved the best results for the left Mastcam images. Moreover, subjective evaluation of five demosaiced Mastcam images was also used to compare the various algorithms.

Original languageEnglish (US)
Article number308
JournalElectronics (Switzerland)
Volume8
Issue number3
DOIs
StatePublished - Mar 1 2019

Fingerprint

Cameras
Color
Deep learning
Fasteners
Digital cameras
Imaging systems
Learning algorithms
Image quality
NASA
Experiments

Keywords

  • Curiosity rover
  • Debayering
  • Deep learning
  • Demosaicing
  • Fusion
  • Image enhancement
  • Mastcam

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Hardware and Architecture
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Cite this

Comparison of deep learning and conventional demosaicing algorithms for mastcam images. / Kwan, Chiman; Chou, Bryan; Bell, James.

In: Electronics (Switzerland), Vol. 8, No. 3, 308, 01.03.2019.

Research output: Contribution to journalArticle

@article{4c6ca4d00be64cdb9197e906912b6d31,
title = "Comparison of deep learning and conventional demosaicing algorithms for mastcam images",
abstract = "Bayer pattern filters have been used in many commercial digital cameras. In National Aeronautics and Space Administration’s (NASA) mast camera (Mastcam) imaging system, onboard the Mars Science Laboratory (MSL) rover Curiosity, a Bayer pattern filter is being used to capture the RGB (red, green, and blue) color of scenes on Mars. The Mastcam has two cameras: left and right. The right camera has three times better resolution than that of the left. It is well known that demosaicing introduces color and zipper artifacts. Here, we present a comparative study of demosaicing results using conventional and deep learning algorithms. Sixteen left and 15 right Mastcam images were used in our experiments. Due to a lack of ground truth images for Mastcam data from Mars, we compared the various algorithms using a blind image quality assessment model. It was observed that no one algorithm can work the best for all images. In particular, a deep learning-based algorithm worked the best for the right Mastcam images and a conventional algorithm achieved the best results for the left Mastcam images. Moreover, subjective evaluation of five demosaiced Mastcam images was also used to compare the various algorithms.",
keywords = "Curiosity rover, Debayering, Deep learning, Demosaicing, Fusion, Image enhancement, Mastcam",
author = "Chiman Kwan and Bryan Chou and James Bell",
year = "2019",
month = "3",
day = "1",
doi = "10.3390/electronics8030308",
language = "English (US)",
volume = "8",
journal = "Electronics (Switzerland)",
issn = "2079-9292",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "3",

}

TY - JOUR

T1 - Comparison of deep learning and conventional demosaicing algorithms for mastcam images

AU - Kwan, Chiman

AU - Chou, Bryan

AU - Bell, James

PY - 2019/3/1

Y1 - 2019/3/1

N2 - Bayer pattern filters have been used in many commercial digital cameras. In National Aeronautics and Space Administration’s (NASA) mast camera (Mastcam) imaging system, onboard the Mars Science Laboratory (MSL) rover Curiosity, a Bayer pattern filter is being used to capture the RGB (red, green, and blue) color of scenes on Mars. The Mastcam has two cameras: left and right. The right camera has three times better resolution than that of the left. It is well known that demosaicing introduces color and zipper artifacts. Here, we present a comparative study of demosaicing results using conventional and deep learning algorithms. Sixteen left and 15 right Mastcam images were used in our experiments. Due to a lack of ground truth images for Mastcam data from Mars, we compared the various algorithms using a blind image quality assessment model. It was observed that no one algorithm can work the best for all images. In particular, a deep learning-based algorithm worked the best for the right Mastcam images and a conventional algorithm achieved the best results for the left Mastcam images. Moreover, subjective evaluation of five demosaiced Mastcam images was also used to compare the various algorithms.

AB - Bayer pattern filters have been used in many commercial digital cameras. In National Aeronautics and Space Administration’s (NASA) mast camera (Mastcam) imaging system, onboard the Mars Science Laboratory (MSL) rover Curiosity, a Bayer pattern filter is being used to capture the RGB (red, green, and blue) color of scenes on Mars. The Mastcam has two cameras: left and right. The right camera has three times better resolution than that of the left. It is well known that demosaicing introduces color and zipper artifacts. Here, we present a comparative study of demosaicing results using conventional and deep learning algorithms. Sixteen left and 15 right Mastcam images were used in our experiments. Due to a lack of ground truth images for Mastcam data from Mars, we compared the various algorithms using a blind image quality assessment model. It was observed that no one algorithm can work the best for all images. In particular, a deep learning-based algorithm worked the best for the right Mastcam images and a conventional algorithm achieved the best results for the left Mastcam images. Moreover, subjective evaluation of five demosaiced Mastcam images was also used to compare the various algorithms.

KW - Curiosity rover

KW - Debayering

KW - Deep learning

KW - Demosaicing

KW - Fusion

KW - Image enhancement

KW - Mastcam

UR - http://www.scopus.com/inward/record.url?scp=85064051539&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064051539&partnerID=8YFLogxK

U2 - 10.3390/electronics8030308

DO - 10.3390/electronics8030308

M3 - Article

VL - 8

JO - Electronics (Switzerland)

JF - Electronics (Switzerland)

SN - 2079-9292

IS - 3

M1 - 308

ER -