Binary neural network with 16 Mb RRAM macro chip for classification and online training

Shimeng Yu, Zhiwei Li, Pai Yu Chen, Huaqiang Wu, Bin Gao, Deli Wang, Wei Wu, He Qian

Research output: Chapter in Book/Report/Conference proceedingConference contribution

29 Citations (Scopus)

Abstract

On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

Original languageEnglish (US)
Title of host publication2016 IEEE International Electron Devices Meeting, IEDM 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages16.2.1-16.2.4
ISBN (Electronic)9781509039012
DOIs
StatePublished - Jan 31 2017
Event62nd IEEE International Electron Devices Meeting, IEDM 2016 - San Francisco, United States
Duration: Dec 3 2016Dec 7 2016

Other

Other62nd IEEE International Electron Devices Meeting, IEDM 2016
CountryUnited States
CitySan Francisco
Period12/3/1612/7/16

Fingerprint

Macros
education
chips
Neural networks
Image recognition
digits
endurance
neurons
Neurons
emerging
CMOS
Durability
methodology
analogs
computer programs
Data storage equipment
cycles
RRAM

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Materials Chemistry
  • Electrical and Electronic Engineering

Cite this

Yu, S., Li, Z., Chen, P. Y., Wu, H., Gao, B., Wang, D., ... Qian, H. (2017). Binary neural network with 16 Mb RRAM macro chip for classification and online training. In 2016 IEEE International Electron Devices Meeting, IEDM 2016 (pp. 16.2.1-16.2.4). [7838429] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEDM.2016.7838429

Binary neural network with 16 Mb RRAM macro chip for classification and online training. / Yu, Shimeng; Li, Zhiwei; Chen, Pai Yu; Wu, Huaqiang; Gao, Bin; Wang, Deli; Wu, Wei; Qian, He.

2016 IEEE International Electron Devices Meeting, IEDM 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 16.2.1-16.2.4 7838429.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yu, S, Li, Z, Chen, PY, Wu, H, Gao, B, Wang, D, Wu, W & Qian, H 2017, Binary neural network with 16 Mb RRAM macro chip for classification and online training. in 2016 IEEE International Electron Devices Meeting, IEDM 2016., 7838429, Institute of Electrical and Electronics Engineers Inc., pp. 16.2.1-16.2.4, 62nd IEEE International Electron Devices Meeting, IEDM 2016, San Francisco, United States, 12/3/16. https://doi.org/10.1109/IEDM.2016.7838429
Yu S, Li Z, Chen PY, Wu H, Gao B, Wang D et al. Binary neural network with 16 Mb RRAM macro chip for classification and online training. In 2016 IEEE International Electron Devices Meeting, IEDM 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 16.2.1-16.2.4. 7838429 https://doi.org/10.1109/IEDM.2016.7838429
Yu, Shimeng ; Li, Zhiwei ; Chen, Pai Yu ; Wu, Huaqiang ; Gao, Bin ; Wang, Deli ; Wu, Wei ; Qian, He. / Binary neural network with 16 Mb RRAM macro chip for classification and online training. 2016 IEEE International Electron Devices Meeting, IEDM 2016. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 16.2.1-16.2.4
@inproceedings{19517dbecee744a088d6d77bf68d3625,
title = "Binary neural network with 16 Mb RRAM macro chip for classification and online training",
abstract = "On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5{\%} accuracy for both classification and online training, close to ∼97{\%} accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.",
author = "Shimeng Yu and Zhiwei Li and Chen, {Pai Yu} and Huaqiang Wu and Bin Gao and Deli Wang and Wei Wu and He Qian",
year = "2017",
month = "1",
day = "31",
doi = "10.1109/IEDM.2016.7838429",
language = "English (US)",
pages = "16.2.1--16.2.4",
booktitle = "2016 IEEE International Electron Devices Meeting, IEDM 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Binary neural network with 16 Mb RRAM macro chip for classification and online training

AU - Yu, Shimeng

AU - Li, Zhiwei

AU - Chen, Pai Yu

AU - Wu, Huaqiang

AU - Gao, Bin

AU - Wang, Deli

AU - Wu, Wei

AU - Qian, He

PY - 2017/1/31

Y1 - 2017/1/31

N2 - On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

AB - On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

UR - http://www.scopus.com/inward/record.url?scp=85014423571&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014423571&partnerID=8YFLogxK

U2 - 10.1109/IEDM.2016.7838429

DO - 10.1109/IEDM.2016.7838429

M3 - Conference contribution

AN - SCOPUS:85014423571

SP - 16.2.1-16.2.4

BT - 2016 IEEE International Electron Devices Meeting, IEDM 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -