BNN Pruning: Pruning Binary Neural Network Guided by Weight Flipping Frequency

Yixing Li, Fengbo Ren

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A binary neural network (BNN) is a compact form of neural network. Both the weights and activations in BNNs can be binary values, which leads to a significant reduction in both parameter size and computational complexity compared to their full-precision counterparts. Such reductions can directly translate into reduced memory footprint and computation cost in hardware, making BNNs highly suitable for a wide range of hardware accelerators. However, it is unclear whether and how a BNN can be further pruned for ultimate compactness. As both 0s and 1s are non-Trivial in BNNs, it is not proper to adopt any existing pruning method of full-precision networks that interprets 0s as trivial. In this paper, we present a pruning method tailored to BNNs and illustrate that BNNs can be further pruned by using weight flipping frequency as an indicator of sensitivity to accuracy. The experiments performed on the binary versions of a 9-layer Network-in-Network (NIN) and the AlexNet with the CIFAR-10 dataset show that the proposed BNN-pruning method can achieve 20-40% reduction in binary operations with 0.5-1.0% accuracy drop, which leads to a 15-40% runtime speedup on a TitanX GPU.

Original languageEnglish (US)
Title of host publicationProceedings of the 21st International Symposium on Quality Electronic Design, ISQED 2020
PublisherIEEE Computer Society
Pages306-311
Number of pages6
ISBN (Electronic)9781728142074
DOIs
StatePublished - Mar 2020
Event21st International Symposium on Quality Electronic Design, ISQED 2020 - Santa Clara, United States
Duration: Mar 25 2020Mar 26 2020

Publication series

NameProceedings - International Symposium on Quality Electronic Design, ISQED
Volume2020-March
ISSN (Print)1948-3287
ISSN (Electronic)1948-3295

Conference

Conference21st International Symposium on Quality Electronic Design, ISQED 2020
CountryUnited States
CitySanta Clara
Period3/25/203/26/20

Keywords

  • binary
  • Neural network
  • pruning

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'BNN Pruning: Pruning Binary Neural Network Guided by Weight Flipping Frequency'. Together they form a unique fingerprint.

Cite this