Mitigating effects of non-ideal synaptic device characteristics for on-chip learning

Pai Yu Chen, Binbin Lin, I. Ting Wang, Tuo Hung Hou, Jieping Ye, Sarma Vrudhula, Jae-sun Seo, Yu Cao, Shimeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

59 Citations (Scopus)

Abstract

The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ?97 % to ?65 %. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ?95 %, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.

Original languageEnglish (US)
Title of host publication2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages194-199
Number of pages6
ISBN (Print)9781467383882
DOIs
StatePublished - Jan 5 2016
Event34th IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015 - Austin, United States
Duration: Nov 2 2015Nov 6 2015

Other

Other34th IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015
CountryUnited States
CityAustin
Period11/2/1511/6/15

Fingerprint

Learning algorithms
Calibration

Keywords

  • cross-point array
  • machine learning
  • neuromorphic computing
  • resistive memory
  • synaptic device

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Chen, P. Y., Lin, B., Wang, I. T., Hou, T. H., Ye, J., Vrudhula, S., ... Yu, S. (2016). Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. In 2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015 (pp. 194-199). [7372570] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCAD.2015.7372570

Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. / Chen, Pai Yu; Lin, Binbin; Wang, I. Ting; Hou, Tuo Hung; Ye, Jieping; Vrudhula, Sarma; Seo, Jae-sun; Cao, Yu; Yu, Shimeng.

2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015. Institute of Electrical and Electronics Engineers Inc., 2016. p. 194-199 7372570.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chen, PY, Lin, B, Wang, IT, Hou, TH, Ye, J, Vrudhula, S, Seo, J, Cao, Y & Yu, S 2016, Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. in 2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015., 7372570, Institute of Electrical and Electronics Engineers Inc., pp. 194-199, 34th IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015, Austin, United States, 11/2/15. https://doi.org/10.1109/ICCAD.2015.7372570
Chen PY, Lin B, Wang IT, Hou TH, Ye J, Vrudhula S et al. Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. In 2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 194-199. 7372570 https://doi.org/10.1109/ICCAD.2015.7372570
Chen, Pai Yu ; Lin, Binbin ; Wang, I. Ting ; Hou, Tuo Hung ; Ye, Jieping ; Vrudhula, Sarma ; Seo, Jae-sun ; Cao, Yu ; Yu, Shimeng. / Mitigating effects of non-ideal synaptic device characteristics for on-chip learning. 2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 194-199
@inproceedings{430b461986a14051a4f73cfcba42c7d4,
title = "Mitigating effects of non-ideal synaptic device characteristics for on-chip learning",
abstract = "The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ?97 {\%} to ?65 {\%}. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ?95 {\%}, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.",
keywords = "cross-point array, machine learning, neuromorphic computing, resistive memory, synaptic device",
author = "Chen, {Pai Yu} and Binbin Lin and Wang, {I. Ting} and Hou, {Tuo Hung} and Jieping Ye and Sarma Vrudhula and Jae-sun Seo and Yu Cao and Shimeng Yu",
year = "2016",
month = "1",
day = "5",
doi = "10.1109/ICCAD.2015.7372570",
language = "English (US)",
isbn = "9781467383882",
pages = "194--199",
booktitle = "2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Mitigating effects of non-ideal synaptic device characteristics for on-chip learning

AU - Chen, Pai Yu

AU - Lin, Binbin

AU - Wang, I. Ting

AU - Hou, Tuo Hung

AU - Ye, Jieping

AU - Vrudhula, Sarma

AU - Seo, Jae-sun

AU - Cao, Yu

AU - Yu, Shimeng

PY - 2016/1/5

Y1 - 2016/1/5

N2 - The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ?97 % to ?65 %. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ?95 %, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.

AB - The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ?97 % to ?65 %. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ?95 %, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.

KW - cross-point array

KW - machine learning

KW - neuromorphic computing

KW - resistive memory

KW - synaptic device

UR - http://www.scopus.com/inward/record.url?scp=84964556162&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964556162&partnerID=8YFLogxK

U2 - 10.1109/ICCAD.2015.7372570

DO - 10.1109/ICCAD.2015.7372570

M3 - Conference contribution

AN - SCOPUS:84964556162

SN - 9781467383882

SP - 194

EP - 199

BT - 2015 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -