TY - GEN
T1 - RACKNet
T2 - 2019 ACM International Conference on Multimedia Retrieval, ICMR 2019
AU - Garg, Yash
AU - Candan, K. Selçuk
N1 - Publisher Copyright:
© 2019 Association for Computing Machinery.
PY - 2019/6/5
Y1 - 2019/6/5
N2 - Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.
AB - Despite their impressive success when these hyper-parameters are suitably fine-tuned, the design of good network architectures remains an art-form rather than a science: while various search techniques, such as grid-search, have been proposed to find effective hyper-parameter configurations, often these parameters are hand-crafted (or the bounds of the search space are provided by a user). In this paper, we argue, and experimentally show, that we can minimize the need for hand-crafting, by relying on the dataset itself. In particular, we show that the dimensions, distributions, and complexities of localized features extracted from the data can inform the structure of the neural networks and help better allocate limited resources (such as kernels) to the various layers of the network. To achieve this, we first present several hypotheses that link the properties of the localized image features to the CNN and RCNN architectures and then, relying on these hypotheses, present a RACKNet framework which aims to learn multiple hyperparameters by extracting information encoded in the input datasets. Experimental evaluations of RACKNet against major benchmark datasets, such as MNIST, SVHN, CIFAR10, COIL20 and ImageNet, show that RACKNet provides significant improvements in the network design and robustness to change in the network.
KW - Convolutional neural networks
KW - Deep learning
KW - Hyper-parameter optimization
KW - Meta-learning
KW - Recurrent neural networks
UR - http://www.scopus.com/inward/record.url?scp=85068063169&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85068063169&partnerID=8YFLogxK
U2 - 10.1145/3323873.3325057
DO - 10.1145/3323873.3325057
M3 - Conference contribution
AN - SCOPUS:85068063169
T3 - ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval
SP - 315
EP - 323
BT - ICMR 2019 - Proceedings of the 2019 ACM International Conference on Multimedia Retrieval
PB - Association for Computing Machinery, Inc
Y2 - 10 June 2019 through 13 June 2019
ER -