A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Deepak Kadetotad; Visar Berisha; Chaitali Chakrabarti; Jae Sun Seo

doi:10.1109/ESSCIRC.2019.8902809

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Deepak Kadetotad, Visar Berisha, Chaitali Chakrabarti, Jae Sun Seo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

Original language	English (US)
Title of host publication	ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	119-122
Number of pages	4
ISBN (Electronic)	9781728115504
DOIs	https://doi.org/10.1109/ESSCIRC.2019.8902809
State	Published - Sep 2019
Event	45th IEEE European Solid State Circuits Conference, ESSCIRC 2019 - Cracow, Poland Duration: Sep 23 2019 → Sep 26 2019

Publication series

Name	ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

Conference

Conference	45th IEEE European Solid State Circuits Conference, ESSCIRC 2019
Country	Poland
City	Cracow
Period	9/23/19 → 9/26/19

Fingerprint

Keywords

Hardware accelerator
long short-term memory (LSTM)
speech recognition
structured sparsity weight compression

ASJC Scopus subject areas

Instrumentation
Electronic, Optical and Magnetic Materials
Hardware and Architecture
Electrical and Electronic Engineering

Cite this

Kadetotad, D., Berisha, V., Chakrabarti, C., & Seo, J. S. (2019). A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference (pp. 119-122). [8902809] (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESSCIRC.2019.8902809

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. / Kadetotad, Deepak; Berisha, Visar ; Chakrabarti, Chaitali ; Seo, Jae Sun.

ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 119-122 8902809 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kadetotad, D, Berisha, V , Chakrabarti, C & Seo, JS 2019, A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. in ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference., 8902809, ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference, Institute of Electrical and Electronics Engineers Inc., pp. 119-122, 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019, Cracow, Poland, 9/23/19. https://doi.org/10.1109/ESSCIRC.2019.8902809

Kadetotad D, Berisha V , Chakrabarti C , Seo JS. A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc. 2019. p. 119-122. 8902809. (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). https://doi.org/10.1109/ESSCIRC.2019.8902809

Kadetotad, Deepak ; Berisha, Visar ; Chakrabarti, Chaitali ; Seo, Jae Sun. / A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 119-122 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).

@inproceedings{e63c7c2336a4455c8b6f850cc9a75995,

title = "A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip",

abstract = "Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.",

keywords = "Hardware accelerator, long short-term memory (LSTM), speech recognition, structured sparsity weight compression",

author = "Deepak Kadetotad and Visar Berisha and Chaitali Chakrabarti and Seo, {Jae Sun}",

year = "2019",

month = sep,

doi = "10.1109/ESSCIRC.2019.8902809",

language = "English (US)",

series = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "119--122",

booktitle = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",

note = "45th IEEE European Solid State Circuits Conference, ESSCIRC 2019 ; Conference date: 23-09-2019 Through 26-09-2019",

}

TY - GEN

T1 - A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

AU - Kadetotad, Deepak

AU - Berisha, Visar

AU - Chakrabarti, Chaitali

AU - Seo, Jae Sun

PY - 2019/9

Y1 - 2019/9

N2 - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

AB - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

KW - Hardware accelerator

KW - long short-term memory (LSTM)

KW - speech recognition

KW - structured sparsity weight compression

UR - http://www.scopus.com/inward/record.url?scp=85075908969&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075908969&partnerID=8YFLogxK

U2 - 10.1109/ESSCIRC.2019.8902809

DO - 10.1109/ESSCIRC.2019.8902809

M3 - Conference contribution

AN - SCOPUS:85075908969

T3 - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

SP - 119

EP - 122

BT - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019

Y2 - 23 September 2019 through 26 September 2019

ER -

Access to Document

10.1109/ESSCIRC.2019.8902809