Parallelizing SRAM arrays with customized bit-cell for binary neural networks

Rui Liu, Xiaochen Peng, Xiaoyu Sun, Win San Khwa, Xin Si, Jia Jing Chen, Jia Fang Li, Meng Fan Chang, Shimeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Recent advances in deep neural networks (DNNs) have shown Binary Neural Networks (BNNs) are able to provide a reasonable accuracy on various image datasets with a significant reduction in computation and memory cost. In this paper, we explore two BNNs: Hybrid BNN (HBNN) and XNORBNN, where the weights are binarized to +1/-1 while the neuron activations are binarized to 1/0 and +1/-1, respectively. Two SRAM bit cell designs are proposed, namely, 6T SRAM for HBNN and customized 8T SRAM for XNOR-BNN. In our design, the high-precision multiply-and-accumulate (MAC) is replaced by bitwise multiplication for HBNN or XNOR for XNOR-BNN plus bit-counting operations. To parallelize the weighted sum operation, we activate multiple word lines in the SRAM array simultaneously and digitize the analog voltage developed along the bit line by a multi-level sense amplifier (MLSA). In order to partition the large matrices in DNNs, we investigate the impact of sensing bit-levels of MLSA on the accuracy degradation for different sub-array sizes and propose using the nonlinear quantization technique to mitigate the accuracy degradation. With 64×64 sub-array size and 3-bit MLSA, HBNN and XNORBNN architectures can minimize the accuracy degradation to 2.37% and 0.88%, respectively, for an inspired VGG-16 network on the CIFAR-10 dataset. Design space exploration of SRAM based synaptic architectures with the conventional row-by-row access scheme and our proposed parallel access scheme are also performed, showing significant benefits in the area, latency and energy-efficiency. Finally, we have successfully taped-out and validated the proposed HBNN and XNOR-BNN designs in TSMC 65 nm process with measured silicon data, achieving energyefficiency >100 TOPS/W for HBNN and >50 TOPS/W for XNOR-BNN.

Original languageEnglish (US)
Title of host publicationProceedings of the 55th Annual Design Automation Conference, DAC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781450357005
DOIs
StatePublished - Jun 24 2018
Event55th Annual Design Automation Conference, DAC 2018 - San Francisco, United States
Duration: Jun 24 2018Jun 29 2018

Publication series

NameProceedings - Design Automation Conference
VolumePart F137710
ISSN (Print)0738-100X

Other

Other55th Annual Design Automation Conference, DAC 2018
CountryUnited States
CitySan Francisco
Period6/24/186/29/18

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Modeling and Simulation

Fingerprint Dive into the research topics of 'Parallelizing SRAM arrays with customized bit-cell for binary neural networks'. Together they form a unique fingerprint.

  • Cite this

    Liu, R., Peng, X., Sun, X., Khwa, W. S., Si, X., Chen, J. J., Li, J. F., Chang, M. F., & Yu, S. (2018). Parallelizing SRAM arrays with customized bit-cell for binary neural networks. In Proceedings of the 55th Annual Design Automation Conference, DAC 2018 [a21] (Proceedings - Design Automation Conference; Vol. Part F137710). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3195970.3196089