XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks

Xiaoyu Sun, Shihui Yin, Xiaochen Peng, Rui Liu, Jae-sun Seo, Shimeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)

Abstract

Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ∼33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

Original languageEnglish (US)
Title of host publicationProceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1423-1428
Number of pages6
Volume2018-January
ISBN (Electronic)9783981926316
DOIs
StatePublished - Apr 19 2018
Event2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 - Dresden, Germany
Duration: Mar 19 2018Mar 23 2018

Other

Other2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
CountryGermany
CityDresden
Period3/19/183/23/18

Fingerprint

Neural networks
Convolution
Energy efficiency
RRAM
Chemical activation
Data storage equipment
Degradation
Costs

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Hardware and Architecture
  • Software
  • Information Systems and Management

Cite this

Sun, X., Yin, S., Peng, X., Liu, R., Seo, J., & Yu, S. (2018). XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks. In Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 (Vol. 2018-January, pp. 1423-1428). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/DATE.2018.8342235

XNOR-RRAM : A scalable and parallel resistive synaptic architecture for binary neural networks. / Sun, Xiaoyu; Yin, Shihui; Peng, Xiaochen; Liu, Rui; Seo, Jae-sun; Yu, Shimeng.

Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 1423-1428.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, X, Yin, S, Peng, X, Liu, R, Seo, J & Yu, S 2018, XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks. in Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 1423-1428, 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018, Dresden, Germany, 3/19/18. https://doi.org/10.23919/DATE.2018.8342235
Sun X, Yin S, Peng X, Liu R, Seo J, Yu S. XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks. In Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1423-1428 https://doi.org/10.23919/DATE.2018.8342235
Sun, Xiaoyu ; Yin, Shihui ; Peng, Xiaochen ; Liu, Rui ; Seo, Jae-sun ; Yu, Shimeng. / XNOR-RRAM : A scalable and parallel resistive synaptic architecture for binary neural networks. Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1423-1428
@inproceedings{2493b5ff0ef74f49b8ed260b10e5c52e,
title = "XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks",
abstract = "Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43{\%} for MLP on MNIST and 86.08{\%} for CNN on CIFAR-10, showing 0.34{\%} and 2.39{\%} degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ∼33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.",
author = "Xiaoyu Sun and Shihui Yin and Xiaochen Peng and Rui Liu and Jae-sun Seo and Shimeng Yu",
year = "2018",
month = "4",
day = "19",
doi = "10.23919/DATE.2018.8342235",
language = "English (US)",
volume = "2018-January",
pages = "1423--1428",
booktitle = "Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - XNOR-RRAM

T2 - A scalable and parallel resistive synaptic architecture for binary neural networks

AU - Sun, Xiaoyu

AU - Yin, Shihui

AU - Peng, Xiaochen

AU - Liu, Rui

AU - Seo, Jae-sun

AU - Yu, Shimeng

PY - 2018/4/19

Y1 - 2018/4/19

N2 - Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ∼33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

AB - Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ∼33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

UR - http://www.scopus.com/inward/record.url?scp=85042001226&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042001226&partnerID=8YFLogxK

U2 - 10.23919/DATE.2018.8342235

DO - 10.23919/DATE.2018.8342235

M3 - Conference contribution

AN - SCOPUS:85042001226

VL - 2018-January

SP - 1423

EP - 1428

BT - Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -