TY - GEN
T1 - MetaGater
T2 - 18th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021
AU - Lin, Sen
AU - Yang, Li
AU - He, Zhezhi
AU - Fan, Deliang
AU - Zhang, Junshan
N1 - Funding Information:
ACKNOWLEDGEMENT This work is supported in part by NSF Grants CNS-2003081, CPS-1739344 and SaTC-1618768.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - There has recently been an increasing interest in computationally-efficient learning methods for resource-constrained applications, e.g., pruning, quantization and channel gating. In this work, we advocate a holistic approach to jointly train the backbone network and the channel gating which can speed up subnet selection for a new task at the resource-limited node. In particular, we develop a federated meta-learning algorithm to jointly train good meta-initializations for both the backbone networks and gating modules, by leveraging the model similarity across learning tasks on different nodes. In this way, the learnt meta-gating module effectively captures the important filters of a good meta-backbone network, and a task-specific conditional channel gated network can be quickly adapted from the meta-initializations using data samples of the new task. The convergence of the proposed federated meta-learning algorithm is established under mild conditions. Experimental results corroborate the effectiveness of our method in comparison to related work.
AB - There has recently been an increasing interest in computationally-efficient learning methods for resource-constrained applications, e.g., pruning, quantization and channel gating. In this work, we advocate a holistic approach to jointly train the backbone network and the channel gating which can speed up subnet selection for a new task at the resource-limited node. In particular, we develop a federated meta-learning algorithm to jointly train good meta-initializations for both the backbone networks and gating modules, by leveraging the model similarity across learning tasks on different nodes. In this way, the learnt meta-gating module effectively captures the important filters of a good meta-backbone network, and a task-specific conditional channel gated network can be quickly adapted from the meta-initializations using data samples of the new task. The convergence of the proposed federated meta-learning algorithm is established under mild conditions. Experimental results corroborate the effectiveness of our method in comparison to related work.
UR - http://www.scopus.com/inward/record.url?scp=85123911559&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85123911559&partnerID=8YFLogxK
U2 - 10.1109/MASS52906.2021.00031
DO - 10.1109/MASS52906.2021.00031
M3 - Conference contribution
AN - SCOPUS:85123911559
T3 - Proceedings - 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021
SP - 164
EP - 172
BT - Proceedings - 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 4 October 2021 through 7 October 2021
ER -