Abstract

The resistive cross-point array architecture has been proposed for on-chip implementation of weighted sum and weight update operations in neuro-inspired learning algorithms. However, several limiting factors potentially hamper the learning accuracy, including the nonlinearity and device variations in weight update, and the read noise, limited ON/OFF weight ratio and array parasitics in weighted sum. With unsupervised sparse coding as a case study algorithm, this paper employs device-Algorithm co-design methodologies to quantify and mitigate the impact of these non-ideal properties on the accuracy. Our analysis shows that the realistic properties in weight update are tolerable, while those in weighted sum are detrimental to the accuracy. With calibration of realistic synaptic behaviors from experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∼96 to ∼30 percent. The strategies to mitigate this accuracy loss include 1) redundant cells to alleviate the impact of device variations; 2) a dummy column to eliminate the off-state current; and 3) selector and larger wire width to reduce IR drop along interconnects. The selector also reduces the leakage power in weight update. With improved properties by these strategies, the accuracy increases back to ∼95 percent, enabling reliable integration of realistic synaptic devices in neuromorphic systems.

Original languageEnglish (US)
Article number7536617
Pages (from-to)257-264
Number of pages8
JournalIEEE Transactions on Multi-Scale Computing Systems
Volume2
Issue number4
DOIs
StatePublished - Oct 1 2016

Fingerprint

Learning algorithms
Wire
Calibration

Keywords

  • cross-point array
  • Machine learning
  • neuro-inspired computing
  • resistive memory
  • selector
  • synaptic device

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Information Systems
  • Hardware and Architecture

Cite this

Design of Resistive Synaptic Array for Implementing On-Chip Sparse Learning. / Chen, Pai Yu; Gao, Ligang; Yu, Shimeng.

In: IEEE Transactions on Multi-Scale Computing Systems, Vol. 2, No. 4, 7536617, 01.10.2016, p. 257-264.

Research output: Contribution to journalArticle

Chen, Pai Yu ; Gao, Ligang ; Yu, Shimeng. / Design of Resistive Synaptic Array for Implementing On-Chip Sparse Learning. In: IEEE Transactions on Multi-Scale Computing Systems. 2016 ; Vol. 2, No. 4. pp. 257-264.
@article{c7dd149f3aeb4a5cb04df5600cf78e6e,
title = "Design of Resistive Synaptic Array for Implementing On-Chip Sparse Learning",
abstract = "The resistive cross-point array architecture has been proposed for on-chip implementation of weighted sum and weight update operations in neuro-inspired learning algorithms. However, several limiting factors potentially hamper the learning accuracy, including the nonlinearity and device variations in weight update, and the read noise, limited ON/OFF weight ratio and array parasitics in weighted sum. With unsupervised sparse coding as a case study algorithm, this paper employs device-Algorithm co-design methodologies to quantify and mitigate the impact of these non-ideal properties on the accuracy. Our analysis shows that the realistic properties in weight update are tolerable, while those in weighted sum are detrimental to the accuracy. With calibration of realistic synaptic behaviors from experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∼96 to ∼30 percent. The strategies to mitigate this accuracy loss include 1) redundant cells to alleviate the impact of device variations; 2) a dummy column to eliminate the off-state current; and 3) selector and larger wire width to reduce IR drop along interconnects. The selector also reduces the leakage power in weight update. With improved properties by these strategies, the accuracy increases back to ∼95 percent, enabling reliable integration of realistic synaptic devices in neuromorphic systems.",
keywords = "cross-point array, Machine learning, neuro-inspired computing, resistive memory, selector, synaptic device",
author = "Chen, {Pai Yu} and Ligang Gao and Shimeng Yu",
year = "2016",
month = "10",
day = "1",
doi = "10.1109/TMSCS.2016.2598742",
language = "English (US)",
volume = "2",
pages = "257--264",
journal = "IEEE Transactions on Multi-Scale Computing Systems",
issn = "2332-7766",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "4",

}

TY - JOUR

T1 - Design of Resistive Synaptic Array for Implementing On-Chip Sparse Learning

AU - Chen, Pai Yu

AU - Gao, Ligang

AU - Yu, Shimeng

PY - 2016/10/1

Y1 - 2016/10/1

N2 - The resistive cross-point array architecture has been proposed for on-chip implementation of weighted sum and weight update operations in neuro-inspired learning algorithms. However, several limiting factors potentially hamper the learning accuracy, including the nonlinearity and device variations in weight update, and the read noise, limited ON/OFF weight ratio and array parasitics in weighted sum. With unsupervised sparse coding as a case study algorithm, this paper employs device-Algorithm co-design methodologies to quantify and mitigate the impact of these non-ideal properties on the accuracy. Our analysis shows that the realistic properties in weight update are tolerable, while those in weighted sum are detrimental to the accuracy. With calibration of realistic synaptic behaviors from experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∼96 to ∼30 percent. The strategies to mitigate this accuracy loss include 1) redundant cells to alleviate the impact of device variations; 2) a dummy column to eliminate the off-state current; and 3) selector and larger wire width to reduce IR drop along interconnects. The selector also reduces the leakage power in weight update. With improved properties by these strategies, the accuracy increases back to ∼95 percent, enabling reliable integration of realistic synaptic devices in neuromorphic systems.

AB - The resistive cross-point array architecture has been proposed for on-chip implementation of weighted sum and weight update operations in neuro-inspired learning algorithms. However, several limiting factors potentially hamper the learning accuracy, including the nonlinearity and device variations in weight update, and the read noise, limited ON/OFF weight ratio and array parasitics in weighted sum. With unsupervised sparse coding as a case study algorithm, this paper employs device-Algorithm co-design methodologies to quantify and mitigate the impact of these non-ideal properties on the accuracy. Our analysis shows that the realistic properties in weight update are tolerable, while those in weighted sum are detrimental to the accuracy. With calibration of realistic synaptic behaviors from experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∼96 to ∼30 percent. The strategies to mitigate this accuracy loss include 1) redundant cells to alleviate the impact of device variations; 2) a dummy column to eliminate the off-state current; and 3) selector and larger wire width to reduce IR drop along interconnects. The selector also reduces the leakage power in weight update. With improved properties by these strategies, the accuracy increases back to ∼95 percent, enabling reliable integration of realistic synaptic devices in neuromorphic systems.

KW - cross-point array

KW - Machine learning

KW - neuro-inspired computing

KW - resistive memory

KW - selector

KW - synaptic device

UR - http://www.scopus.com/inward/record.url?scp=85008498873&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85008498873&partnerID=8YFLogxK

U2 - 10.1109/TMSCS.2016.2598742

DO - 10.1109/TMSCS.2016.2598742

M3 - Article

VL - 2

SP - 257

EP - 264

JO - IEEE Transactions on Multi-Scale Computing Systems

JF - IEEE Transactions on Multi-Scale Computing Systems

SN - 2332-7766

IS - 4

M1 - 7536617

ER -