Abstract

We study deep neural networks for classification of images with quality distortions. Deep network performance on poor quality images can be greatly improved if the network is fine-tuned with distorted data. However, it is difficult for a single fine-tuned network to perform well across multiple distortion types. We propose a mixture of experts-based ensemble method, MixQualNet, that is robust to multiple different types of distortions. The 'experts' in our model are trained on a particular type of distortion. The output of the model is a weighted sum of the expert models, where the weights are determined by a separate gating network. The gating network is trained to predict weights for a particular distortion type and level. During testing, the network is blind to the distortion level and type, yet can still assign appropriate weights to the expert models. In order to reduce the computational complexity, we introduce weight sharing into the MixQualNet. We utilize the TreeNet weight sharing architecture as well as introduce the Inverted TreeNet architecture. While both weight sharing architectures reduce memory requirements, our proposed Inverted TreeNet also achieves improved accuracy.

Original languageEnglish (US)
Article number8410945
Pages (from-to)5553-5562
Number of pages10
JournalIEEE Transactions on Image Processing
Volume27
Issue number11
DOIs
StatePublished - Nov 1 2018

Fingerprint

Memory architecture
Network performance
Image quality
Deep neural networks
Computational complexity
Testing

Keywords

  • convolutional neural networks
  • deep learning
  • human eye fixations
  • Saliency

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

Quality Robust Mixtures of Deep Neural Networks. / Dodge, Samuel F.; Karam, Lina.

In: IEEE Transactions on Image Processing, Vol. 27, No. 11, 8410945, 01.11.2018, p. 5553-5562.

Research output: Contribution to journalArticle

Dodge, Samuel F. ; Karam, Lina. / Quality Robust Mixtures of Deep Neural Networks. In: IEEE Transactions on Image Processing. 2018 ; Vol. 27, No. 11. pp. 5553-5562.
@article{d2341c7059444e92a22cce384d392541,
title = "Quality Robust Mixtures of Deep Neural Networks",
abstract = "We study deep neural networks for classification of images with quality distortions. Deep network performance on poor quality images can be greatly improved if the network is fine-tuned with distorted data. However, it is difficult for a single fine-tuned network to perform well across multiple distortion types. We propose a mixture of experts-based ensemble method, MixQualNet, that is robust to multiple different types of distortions. The 'experts' in our model are trained on a particular type of distortion. The output of the model is a weighted sum of the expert models, where the weights are determined by a separate gating network. The gating network is trained to predict weights for a particular distortion type and level. During testing, the network is blind to the distortion level and type, yet can still assign appropriate weights to the expert models. In order to reduce the computational complexity, we introduce weight sharing into the MixQualNet. We utilize the TreeNet weight sharing architecture as well as introduce the Inverted TreeNet architecture. While both weight sharing architectures reduce memory requirements, our proposed Inverted TreeNet also achieves improved accuracy.",
keywords = "convolutional neural networks, deep learning, human eye fixations, Saliency",
author = "Dodge, {Samuel F.} and Lina Karam",
year = "2018",
month = "11",
day = "1",
doi = "10.1109/TIP.2018.2855966",
language = "English (US)",
volume = "27",
pages = "5553--5562",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "11",

}

TY - JOUR

T1 - Quality Robust Mixtures of Deep Neural Networks

AU - Dodge, Samuel F.

AU - Karam, Lina

PY - 2018/11/1

Y1 - 2018/11/1

N2 - We study deep neural networks for classification of images with quality distortions. Deep network performance on poor quality images can be greatly improved if the network is fine-tuned with distorted data. However, it is difficult for a single fine-tuned network to perform well across multiple distortion types. We propose a mixture of experts-based ensemble method, MixQualNet, that is robust to multiple different types of distortions. The 'experts' in our model are trained on a particular type of distortion. The output of the model is a weighted sum of the expert models, where the weights are determined by a separate gating network. The gating network is trained to predict weights for a particular distortion type and level. During testing, the network is blind to the distortion level and type, yet can still assign appropriate weights to the expert models. In order to reduce the computational complexity, we introduce weight sharing into the MixQualNet. We utilize the TreeNet weight sharing architecture as well as introduce the Inverted TreeNet architecture. While both weight sharing architectures reduce memory requirements, our proposed Inverted TreeNet also achieves improved accuracy.

AB - We study deep neural networks for classification of images with quality distortions. Deep network performance on poor quality images can be greatly improved if the network is fine-tuned with distorted data. However, it is difficult for a single fine-tuned network to perform well across multiple distortion types. We propose a mixture of experts-based ensemble method, MixQualNet, that is robust to multiple different types of distortions. The 'experts' in our model are trained on a particular type of distortion. The output of the model is a weighted sum of the expert models, where the weights are determined by a separate gating network. The gating network is trained to predict weights for a particular distortion type and level. During testing, the network is blind to the distortion level and type, yet can still assign appropriate weights to the expert models. In order to reduce the computational complexity, we introduce weight sharing into the MixQualNet. We utilize the TreeNet weight sharing architecture as well as introduce the Inverted TreeNet architecture. While both weight sharing architectures reduce memory requirements, our proposed Inverted TreeNet also achieves improved accuracy.

KW - convolutional neural networks

KW - deep learning

KW - human eye fixations

KW - Saliency

UR - http://www.scopus.com/inward/record.url?scp=85049946603&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049946603&partnerID=8YFLogxK

U2 - 10.1109/TIP.2018.2855966

DO - 10.1109/TIP.2018.2855966

M3 - Article

VL - 27

SP - 5553

EP - 5562

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 11

M1 - 8410945

ER -