RACKNet

Robust allocation of convolutional kernels in neural networks for image classification

Yash Garg, Kasim Candan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.

Original languageEnglish (US)
Title of host publicationICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages315-323
Number of pages9
ISBN (Electronic)9781450367653
DOIs
StatePublished - Jun 5 2019
Externally publishedYes
Event2019 ACM International Conference on Multimedia Retrieval, ICMR 2019 - Ottawa, Canada
Duration: Jun 10 2019Jun 13 2019

Publication series

NameICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval

Conference

Conference2019 ACM International Conference on Multimedia Retrieval, ICMR 2019
CountryCanada
CityOttawa
Period6/10/196/13/19

Fingerprint

Image classification
Neural networks
Network architecture

Keywords

  • Convolutional neural networks
  • Deep learning
  • Hyper-parameter optimization
  • Meta-learning
  • Recurrent neural networks

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition

Cite this

Garg, Y., & Candan, K. (2019). RACKNet: Robust allocation of convolutional kernels in neural networks for image classification. In ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval (pp. 315-323). (ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval). Association for Computing Machinery, Inc. https://doi.org/10.1145/3323873.3325057

RACKNet : Robust allocation of convolutional kernels in neural networks for image classification. / Garg, Yash; Candan, Kasim.

ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2019. p. 315-323 (ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Garg, Y & Candan, K 2019, RACKNet: Robust allocation of convolutional kernels in neural networks for image classification. in ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval. ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval, Association for Computing Machinery, Inc, pp. 315-323, 2019 ACM International Conference on Multimedia Retrieval, ICMR 2019, Ottawa, Canada, 6/10/19. https://doi.org/10.1145/3323873.3325057
Garg Y, Candan K. RACKNet: Robust allocation of convolutional kernels in neural networks for image classification. In ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc. 2019. p. 315-323. (ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval). https://doi.org/10.1145/3323873.3325057
Garg, Yash ; Candan, Kasim. / RACKNet : Robust allocation of convolutional kernels in neural networks for image classification. ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2019. pp. 315-323 (ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval).
@inproceedings{27842ebefb964617a18d92810952ad4f,
title = "RACKNet: Robust allocation of convolutional kernels in neural networks for image classification",
abstract = "Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.",
keywords = "Convolutional neural networks, Deep learning, Hyper-parameter optimization, Meta-learning, Recurrent neural networks",
author = "Yash Garg and Kasim Candan",
year = "2019",
month = "6",
day = "5",
doi = "10.1145/3323873.3325057",
language = "English (US)",
series = "ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval",
publisher = "Association for Computing Machinery, Inc",
pages = "315--323",
booktitle = "ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval",

}

TY - GEN

T1 - RACKNet

T2 - Robust allocation of convolutional kernels in neural networks for image classification

AU - Garg, Yash

AU - Candan, Kasim

PY - 2019/6/5

Y1 - 2019/6/5

N2 - Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.

AB - Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.

KW - Convolutional neural networks

KW - Deep learning

KW - Hyper-parameter optimization

KW - Meta-learning

KW - Recurrent neural networks

UR - http://www.scopus.com/inward/record.url?scp=85068063169&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068063169&partnerID=8YFLogxK

U2 - 10.1145/3323873.3325057

DO - 10.1145/3323873.3325057

M3 - Conference contribution

T3 - ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval

SP - 315

EP - 323

BT - ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval

PB - Association for Computing Machinery, Inc

ER -