TY - GEN
T1 - WSNet
T2 - 35th International Conference on Machine Learning, ICML 2018
AU - Jin, Xiaojie
AU - Yang, Yingzhen
AU - Xu, Ning
AU - Yang, Jianchao
AU - Jojic, Nebojsa
AU - Feng, Jiashi
AU - Yan, Shuicheng
N1 - Funding Information:
Jiashi Feng was partially supported by NUS startup R-263-000-C08-133, MOE Tier-I R-263-000-C21-112, NUS IDS R-263-000-C67-646, ECRA R-263-000-C87-133 and MOE Tier-II R-263-000-D17-112.
Publisher Copyright:
© 2018 by authors.All right reserved.
PY - 2018
Y1 - 2018
N2 - We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks. Existing approaches conventionally learn full model parameters independently and then compress them via ad hoc processing such as model pruning or filter factorization. Alternatively, WSNet proposes learning model parameters by sampling from a compact set of learnable parameters, which naturally enforces parameter sharing throughout the learning process. We demonstrate that such a novel weight sampling approach (and induced WSNet) promotes both weights and computation sharing favorably. By employing this method, we can more efficiently learn much smaller networks with competitive performance compared to baseline networks with equal numbers of convolution filters. Specifically, we consider learning compact and efficient 1D convolutional neural networks for audio classification. Extensive experiments on multiple audio classification datasets verify the effectiveness of WSNet. Combined with weight quantization, the resulted models are up to 180× smaller and theoretically up to 16× faster than the well-established baselines, without noticeable performance drop.
AB - We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks. Existing approaches conventionally learn full model parameters independently and then compress them via ad hoc processing such as model pruning or filter factorization. Alternatively, WSNet proposes learning model parameters by sampling from a compact set of learnable parameters, which naturally enforces parameter sharing throughout the learning process. We demonstrate that such a novel weight sampling approach (and induced WSNet) promotes both weights and computation sharing favorably. By employing this method, we can more efficiently learn much smaller networks with competitive performance compared to baseline networks with equal numbers of convolution filters. Specifically, we consider learning compact and efficient 1D convolutional neural networks for audio classification. Extensive experiments on multiple audio classification datasets verify the effectiveness of WSNet. Combined with weight quantization, the resulted models are up to 180× smaller and theoretically up to 16× faster than the well-established baselines, without noticeable performance drop.
UR - http://www.scopus.com/inward/record.url?scp=85057242887&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85057242887&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85057242887
T3 - 35th International Conference on Machine Learning, ICML 2018
SP - 3683
EP - 3696
BT - 35th International Conference on Machine Learning, ICML 2018
A2 - Dy, Jennifer
A2 - Krause, Andreas
PB - International Machine Learning Society (IMLS)
Y2 - 10 July 2018 through 15 July 2018
ER -