Abstract

Recurrent neural networks (RNNs) provide excellent performance on applications with sequential data such as speech recognition. On-chip implementation of RNNs is difficult due to the significantly large number of parameters and computations. In this work, we first present a training method for LSTM model for language modeling on Penn Treebank dataset with binary weights and multi-bit activations and then map it onto a fully parallel RRAM array architecture ("XNOR-RRAM"). An energy-efficient XNOR-RRAM array based system for LSTM RNN is implemented and benchmarked on Penn Treebank dataset. Our results show that 4-bit activation precision can provide a near-optimal perplexity of 115.3 with an estimated energy-efficiency of 27 TOPS/W.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE Workshop on Signal Processing Systems, SiPS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages13-18
Number of pages6
ISBN (Electronic)9781538663189
DOIs
StatePublished - Dec 31 2018
Event2018 IEEE Workshop on Signal Processing Systems, SiPS 2018 - Cape Town, South Africa
Duration: Oct 21 2018Oct 24 2018

Publication series

NameIEEE Workshop on Signal Processing Systems, SiPS: Design and Implementation
Volume2018-October
ISSN (Print)1520-6130

Conference

Conference2018 IEEE Workshop on Signal Processing Systems, SiPS 2018
CountrySouth Africa
CityCape Town
Period10/21/1810/24/18

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing
  • Applied Mathematics
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'A Parallel RRAM Synaptic Array Architecture for Energy-Efficient Recurrent Neural Networks'. Together they form a unique fingerprint.

  • Cite this

    Yin, S., Sun, X., Yu, S., Seo, J., & Chakrabarti, C. (2018). A Parallel RRAM Synaptic Array Architecture for Energy-Efficient Recurrent Neural Networks. In Proceedings of the IEEE Workshop on Signal Processing Systems, SiPS 2018 (pp. 13-18). [8598445] (IEEE Workshop on Signal Processing Systems, SiPS: Design and Implementation; Vol. 2018-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SiPS.2018.8598445