Accurate Inference with Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

Gouranga Charan, Abinash Mohanty, Xiaocong Du, Gokul Krishnan, Rajiv V. Joshi, Yu Cao

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Resistive random access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pretrained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from accuracy degradation due to RRAM nonidealities, such as device variations, quantization error, and stuck-at-faults. Previous solutions involving multiple read-verify-write (R-V-W) to the RRAM cells require cell-by-cell compensation and, thus, an excessive amount of processing time. In this article, we propose a joint algorithm-design solution to mitigate the accuracy degradation. We first leverage knowledge distillation (KD), where the model is trained with the RRAM nonidealities to increase the robustness of the model under device variations. Furthermore, we propose random sparse adaptation (RSA), which integrates a small on-chip memory with the main RRAM array for postmapping adaptation. Only the on-chip memory is updated to recover the inference accuracy. The joint algorithm-design solution achieves the state-of-the-art accuracy of 99.41% for MNIST (LeNet-5) and 91.86% for CIFAR-10 (VGG-16) with up to 5% parameters as overhead while providing a 15-150× speedup compared with R-V-W.

Original languageEnglish (US)
Article number9069242
Pages (from-to)27-35
Number of pages9
JournalIEEE Journal on Exploratory Solid-State Computational Devices and Circuits
Volume6
Issue number1
DOIs
StatePublished - Jun 2020

Keywords

  • Convolution neural networks
  • device nonidealities
  • model robustness
  • neuromorphic computing
  • random sparse adaptation (RSA)
  • resistive random access memory (RRAM)

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Accurate Inference with Inaccurate RRAM Devices: A Joint Algorithm-Design Solution'. Together they form a unique fingerprint.

Cite this