XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks

Xiaoyu Sun, Shihui Yin, Xiaochen Peng, Rui Liu, Jae-sun Seo, Shimeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

43 Scopus citations

Abstract

Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ∼33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

Original languageEnglish (US)
Title of host publicationProceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1423-1428
Number of pages6
Volume2018-January
ISBN (Electronic)9783981926316
DOIs
StatePublished - Apr 19 2018
Event2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 - Dresden, Germany
Duration: Mar 19 2018Mar 23 2018

Other

Other2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
CountryGermany
CityDresden
Period3/19/183/23/18

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Hardware and Architecture
  • Software
  • Information Systems and Management

Fingerprint Dive into the research topics of 'XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks'. Together they form a unique fingerprint.

  • Cite this

    Sun, X., Yin, S., Peng, X., Liu, R., Seo, J., & Yu, S. (2018). XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks. In Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 (Vol. 2018-January, pp. 1423-1428). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/DATE.2018.8342235