A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Deepak Kadetotad, Visar Berisha, Chaitali Chakrabarti, Jae Sun Seo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

Original languageEnglish (US)
Title of host publicationESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages119-122
Number of pages4
ISBN (Electronic)9781728115504
DOIs
StatePublished - Sep 2019
Event45th IEEE European Solid State Circuits Conference, ESSCIRC 2019 - Cracow, Poland
Duration: Sep 23 2019Sep 26 2019

Publication series

NameESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

Conference

Conference45th IEEE European Solid State Circuits Conference, ESSCIRC 2019
CountryPoland
CityCracow
Period9/23/199/26/19

Fingerprint

TOPS (spacecraft)
Recurrent neural networks
Particle accelerators
accelerators
chips
hardware
transferred electron devices
Computer hardware
CMOS
Hardware
Data storage equipment
prototypes
Long short-term memory
TOPS
requirements

Keywords

  • Hardware accelerator
  • long short-term memory (LSTM)
  • speech recognition
  • structured sparsity weight compression

ASJC Scopus subject areas

  • Instrumentation
  • Electronic, Optical and Magnetic Materials
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Kadetotad, D., Berisha, V., Chakrabarti, C., & Seo, J. S. (2019). A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference (pp. 119-122). [8902809] (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESSCIRC.2019.8902809

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. / Kadetotad, Deepak; Berisha, Visar; Chakrabarti, Chaitali; Seo, Jae Sun.

ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 119-122 8902809 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kadetotad, D, Berisha, V, Chakrabarti, C & Seo, JS 2019, A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. in ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference., 8902809, ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference, Institute of Electrical and Electronics Engineers Inc., pp. 119-122, 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019, Cracow, Poland, 9/23/19. https://doi.org/10.1109/ESSCIRC.2019.8902809
Kadetotad D, Berisha V, Chakrabarti C, Seo JS. A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc. 2019. p. 119-122. 8902809. (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). https://doi.org/10.1109/ESSCIRC.2019.8902809
Kadetotad, Deepak ; Berisha, Visar ; Chakrabarti, Chaitali ; Seo, Jae Sun. / A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 119-122 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).
@inproceedings{e63c7c2336a4455c8b6f850cc9a75995,
title = "A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip",
abstract = "Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.",
keywords = "Hardware accelerator, long short-term memory (LSTM), speech recognition, structured sparsity weight compression",
author = "Deepak Kadetotad and Visar Berisha and Chaitali Chakrabarti and Seo, {Jae Sun}",
year = "2019",
month = "9",
doi = "10.1109/ESSCIRC.2019.8902809",
language = "English (US)",
series = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "119--122",
booktitle = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",

}

TY - GEN

T1 - A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

AU - Kadetotad, Deepak

AU - Berisha, Visar

AU - Chakrabarti, Chaitali

AU - Seo, Jae Sun

PY - 2019/9

Y1 - 2019/9

N2 - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

AB - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

KW - Hardware accelerator

KW - long short-term memory (LSTM)

KW - speech recognition

KW - structured sparsity weight compression

UR - http://www.scopus.com/inward/record.url?scp=85075908969&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075908969&partnerID=8YFLogxK

U2 - 10.1109/ESSCIRC.2019.8902809

DO - 10.1109/ESSCIRC.2019.8902809

M3 - Conference contribution

AN - SCOPUS:85075908969

T3 - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

SP - 119

EP - 122

BT - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

PB - Institute of Electrical and Electronics Engineers Inc.

ER -