Robust sparse regularization: Defending adversarial attacks via regularized sparse network

Adrian Siraj Rakin, Zhezhi He, Li Yang, Yanzhi Wang, Liqiang Wang, Deliang Fan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep Neural Network (DNN) trained by the gradient descent method is known to be vulnerable to maliciously perturbed adversarial input, aka. adversarial attack. As one of the countermeasures against adversarial attacks, increasing the model capacity for DNN robustness enhancement was discussed and reported as an effective approach by many recent works. In this work, we show that shrinking the model size through proper weight pruning can even be helpful to improve the DNN robustness under adversarial attack. For obtaining a simultaneously robust and compact DNN model, we propose a multi-objective training method called Robust Sparse Regularization (RSR), through the fusion of various regularization techniques, including channel-wise noise injection, lasso weight penalty, and adversarial training. We conduct extensive experiments to show the effectiveness of RSR against popular white-box (i.e., PGD and FGSM) and black-box attacks. Thanks to RSR, 85% weight connections of ResNet-18 can be pruned while still achieving 0.68% and 8.72% improvement in clean- and perturbed-data accuracy respectively on CIFAR-10 dataset, in comparison to its PGD adversarial training baseline.

Original languageEnglish (US)
Title of host publicationGLSVLSI 2020 - Proceedings of the 2020 Great Lakes Symposium on VLSI
PublisherAssociation for Computing Machinery
Pages125-130
Number of pages6
ISBN (Electronic)9781450379441
DOIs
StatePublished - Sep 7 2020
Event30th Great Lakes Symposium on VLSI, GLSVLSI 2020 - Virtual, Online, China
Duration: Sep 7 2020Sep 9 2020

Publication series

NameProceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI

Conference

Conference30th Great Lakes Symposium on VLSI, GLSVLSI 2020
CountryChina
CityVirtual, Online
Period9/7/209/9/20

Keywords

  • Adversarial Defense
  • Robust
  • Sparse

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Robust sparse regularization: Defending adversarial attacks via regularized sparse network'. Together they form a unique fingerprint.

Cite this