Random sparse adaptation for accurate inference with inaccurate multi-level RRAM arrays

Abinash Mohanty, Xiaocong Du, Pai Yu Chen, Jae-sun Seo, Shimeng Yu, Yu Cao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

An array of multi-level resistive memory devices (RRAMs) can speed up the computation of deep learning algorithms. However, when a pre-trained model is programmed to a real RRAM array for inference, its accuracy degrades due to many non-idealities, such as variations, quantization error, and stuck-at faults. A conventional solution involves multiple read-verify-write (R-V-W) for each RRAM cell, costing a long time because of the slow Write speed and cell-by-cell compensation. In this work, we propose a fundamentally new approach to overcome this issue: random sparse adaptation (RSA) after the model is transferred to the RRAM array. By randomly selecting a small portion of model parameters and mapping them to onchip memory for further training, we demonstrate an efficient and fast method to recover the accuracy: in CNNs for MNIST and CIFAR-10, -5% of model parameters is sufficient for RSA even under excessive RRAM variations. As the backpropagation in training is only applied to RSA cells and there is no need of any Write operation on RRAM, the proposed RSA achieves 10-100X acceleration compared to R-V-W. Therefore, this hybrid solution with a large, inaccurate RRAM array and a small, accurate on-chip memory array promises both area efficiency and inference accuracy.

Original languageEnglish (US)
Title of host publication2017 IEEE International Electron Devices Meeting, IEDM 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6.3.1-6.3.4
VolumePart F134366
ISBN (Electronic)9781538635599
DOIs
StatePublished - Jan 23 2018
Event63rd IEEE International Electron Devices Meeting, IEDM 2017 - San Francisco, United States
Duration: Dec 2 2017Dec 6 2017

Other

Other63rd IEEE International Electron Devices Meeting, IEDM 2017
CountryUnited States
CitySan Francisco
Period12/2/1712/6/17

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Electrical and Electronic Engineering
  • Materials Chemistry

Fingerprint Dive into the research topics of 'Random sparse adaptation for accurate inference with inaccurate multi-level RRAM arrays'. Together they form a unique fingerprint.

  • Cite this

    Mohanty, A., Du, X., Chen, P. Y., Seo, J., Yu, S., & Cao, Y. (2018). Random sparse adaptation for accurate inference with inaccurate multi-level RRAM arrays. In 2017 IEEE International Electron Devices Meeting, IEDM 2017 (Vol. Part F134366, pp. 6.3.1-6.3.4). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEDM.2017.8268339