Accurate Inference with Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

Gouranga Charan, Abinash Mohanty, Xiaocong Du, Gokul Krishnan, Rajiv V. Joshi, Yu Cao

Research output: Contribution to journalArticle

Abstract

Resistive random-access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pre-trained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from accuracy degradation due to RRAM nonidealities such as device variations, quantization error, and stuckat- faults. Previous solutions involving multiple read-verify-write (R-V-W) to the RRAM cells, require cell-by-cell compensation, and thus, an excessive amount of processing time. In this paper, we propose a joint algorithm-design solution to mitigate the accuracy degradation: 1)We first leverage Knowledge Distillation (KD), where the model is trained with the RRAM non-idealities to increase the robustness of the model under device variations. 2) Furthermore, we propose random sparse adaptation (RSA), which integrates a small on-chip memory with the main RRAM array for post-mapping adaptation. Only the on-chip memory is updated to recover the inference accuracy. The joint algorithmdesign solution achieves the state-of-the-art accuracy of 99.41% for MNIST (LeNet-5) and 91.86% for CIFAR-10 (VGG-16) with up to 5% parameters as overhead while providing a 15-150X speedup as compared to R-V-W.

    Fingerprint

Keywords

  • convolution neural networks (CNN)
  • device non-idealities
  • model robustness
  • Neuromorphic computing
  • random sparse adaptation
  • Resistive random access memory (RRAM)

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this