MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning

Sen Lin, Li Yang, Zhezhi He, Deliang Fan, Junshan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

There has recently been an increasing interest in computationally-efficient learning methods for resource-constrained applications, e.g., pruning, quantization and channel gating. In this work, we advocate a holistic approach to jointly train the backbone network and the channel gating which can speed up subnet selection for a new task at the resource-limited node. In particular, we develop a federated meta-learning algorithm to jointly train good meta-initializations for both the backbone networks and gating modules, by leveraging the model similarity across learning tasks on different nodes. In this way, the learnt meta-gating module effectively captures the important filters of a good meta-backbone network, and a task-specific conditional channel gated network can be quickly adapted from the meta-initializations using data samples of the new task. The convergence of the proposed federated meta-learning algorithm is established under mild conditions. Experimental results corroborate the effectiveness of our method in comparison to related work.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages164-172
Number of pages9
ISBN (Electronic)9781665449359
DOIs
StatePublished - 2021
Event18th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021 - Virtual, Online, United States
Duration: Oct 4 2021Oct 7 2021

Publication series

NameProceedings - 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021

Conference

Conference18th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2021
Country/TerritoryUnited States
CityVirtual, Online
Period10/4/2110/7/21

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning'. Together they form a unique fingerprint.

Cite this