Binary neural network with 16 Mb RRAM macro chip for classification and online training

Shimeng Yu; Zhiwei Li; Pai Yu Chen; Huaqiang Wu; Bin Gao; Deli Wang; Wei Wu; He Qian

doi:10.1109/IEDM.2016.7838429

Binary neural network with 16 Mb RRAM macro chip for classification and online training

Shimeng Yu, Zhiwei Li, Pai Yu Chen, Huaqiang Wu, Bin Gao, Deli Wang, Wei Wu, He Qian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

88 Scopus citations

Abstract

On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

Original language	English (US)
Title of host publication	2016 IEEE International Electron Devices Meeting, IEDM 2016
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	16.2.1-16.2.4
ISBN (Electronic)	9781509039012
DOIs	https://doi.org/10.1109/IEDM.2016.7838429
State	Published - Jan 31 2017
Event	62nd IEEE International Electron Devices Meeting, IEDM 2016 - San Francisco, United States Duration: Dec 3 2016 → Dec 7 2016

Other

Other	62nd IEEE International Electron Devices Meeting, IEDM 2016
Country/Territory	United States
City	San Francisco
Period	12/3/16 → 12/7/16

ASJC Scopus subject areas

Electronic, Optical and Magnetic Materials
Condensed Matter Physics
Materials Chemistry
Electrical and Electronic Engineering

Access to Document

10.1109/IEDM.2016.7838429

Cite this

Binary neural network with 16 Mb RRAM macro chip for classification and online training. / Yu, Shimeng; Li, Zhiwei; Chen, Pai Yu et al.
2016 IEEE International Electron Devices Meeting, IEDM 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 16.2.1-16.2.4 7838429.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yu, S, Li, Z, Chen, PY, Wu, H, Gao, B, Wang, D, Wu, W & Qian, H 2017, Binary neural network with 16 Mb RRAM macro chip for classification and online training. in 2016 IEEE International Electron Devices Meeting, IEDM 2016., 7838429, Institute of Electrical and Electronics Engineers Inc., pp. 16.2.1-16.2.4, 62nd IEEE International Electron Devices Meeting, IEDM 2016, San Francisco, United States, 12/3/16. https://doi.org/10.1109/IEDM.2016.7838429

@inproceedings{19517dbecee744a088d6d77bf68d3625,

title = "Binary neural network with 16 Mb RRAM macro chip for classification and online training",

abstract = "On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.",

author = "Shimeng Yu and Zhiwei Li and Chen, {Pai Yu} and Huaqiang Wu and Bin Gao and Deli Wang and Wei Wu and He Qian",

year = "2017",

month = jan,

day = "31",

doi = "10.1109/IEDM.2016.7838429",

language = "English (US)",

pages = "16.2.1--16.2.4",

booktitle = "2016 IEEE International Electron Devices Meeting, IEDM 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "62nd IEEE International Electron Devices Meeting, IEDM 2016 ; Conference date: 03-12-2016 Through 07-12-2016",

}

TY - GEN

T1 - Binary neural network with 16 Mb RRAM macro chip for classification and online training

AU - Yu, Shimeng

AU - Li, Zhiwei

AU - Chen, Pai Yu

AU - Wu, Huaqiang

AU - Gao, Bin

AU - Wang, Deli

AU - Wu, Wei

AU - Qian, He

PY - 2017/1/31

Y1 - 2017/1/31

N2 - On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

AB - On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ∼96.5% accuracy for both classification and online training, close to ∼97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

UR - http://www.scopus.com/inward/record.url?scp=85014423571&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014423571&partnerID=8YFLogxK

U2 - 10.1109/IEDM.2016.7838429

DO - 10.1109/IEDM.2016.7838429

M3 - Conference contribution

AN - SCOPUS:85014423571

SP - 16.2.1-16.2.4

BT - 2016 IEEE International Electron Devices Meeting, IEDM 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 62nd IEEE International Electron Devices Meeting, IEDM 2016

Y2 - 3 December 2016 through 7 December 2016

ER -

Binary neural network with 16 Mb RRAM macro chip for classification and online training

Abstract

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this