A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Deepak Kadetotad; Visar Berisha; Chaitali Chakrabarti; Jae Sun Seo

doi:10.1109/lssc.2019.2936761

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Deepak Kadetotad, Visar Berisha, Chaitali Chakrabarti, Jae Sun Seo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

Original language	English (US)
Title of host publication	ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	119-122
Number of pages	4
ISBN (Electronic)	9781728115504
DOIs	https://doi.org/10.1109/lssc.2019.2936761
State	Published - Sep 2019
Event	45th IEEE European Solid State Circuits Conference, ESSCIRC 2019 - Cracow, Poland Duration: Sep 23 2019 → Sep 26 2019

Publication series

Name	ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

Conference

Conference	45th IEEE European Solid State Circuits Conference, ESSCIRC 2019
Country/Territory	Poland
City	Cracow
Period	9/23/19 → 9/26/19

Keywords

Hardware accelerator
long short-term memory (LSTM)
speech recognition
structured sparsity weight compression

ASJC Scopus subject areas

Instrumentation
Electronic, Optical and Magnetic Materials
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/lssc.2019.2936761

Cite this

Kadetotad, D., Berisha, V., Chakrabarti, C., & Seo, J. S. (2019). A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference (pp. 119-122). Article 8902809 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/lssc.2019.2936761

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. / Kadetotad, Deepak; Berisha, Visar ; Chakrabarti, Chaitali et al.
ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 119-122 8902809 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kadetotad, D, Berisha, V , Chakrabarti, C & Seo, JS 2019, A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. in ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference., 8902809, ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference, Institute of Electrical and Electronics Engineers Inc., pp. 119-122, 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019, Cracow, Poland, 9/23/19. https://doi.org/10.1109/lssc.2019.2936761

Kadetotad D, Berisha V , Chakrabarti C, Seo JS. A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. In ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc. 2019. p. 119-122. 8902809. (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference). doi: 10.1109/lssc.2019.2936761

Kadetotad, Deepak ; Berisha, Visar ; Chakrabarti, Chaitali et al. / A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip. ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 119-122 (ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference).

@inproceedings{e63c7c2336a4455c8b6f850cc9a75995,

title = "A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip",

abstract = "Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.",

keywords = "Hardware accelerator, long short-term memory (LSTM), speech recognition, structured sparsity weight compression",

author = "Deepak Kadetotad and Visar Berisha and Chaitali Chakrabarti and Seo, {Jae Sun}",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019 ; Conference date: 23-09-2019 Through 26-09-2019",

year = "2019",

month = sep,

doi = "10.1109/lssc.2019.2936761",

language = "English (US)",

series = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "119--122",

booktitle = "ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference",

}

TY - GEN

T1 - A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

AU - Kadetotad, Deepak

AU - Berisha, Visar

AU - Chakrabarti, Chaitali

AU - Seo, Jae Sun

PY - 2019/9

Y1 - 2019/9

N2 - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

AB - Long short-term memory (LSTM) networks are widely used for speech applications but pose difficulties for efficient implementation on hardware due to large weight storage requirements. We present an energy-efficient LSTM recurrent neural network (RNN) accelerator,featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by HCGS-based block-wise recursive weight compression,we demonstrate LSTM networks with up to 16× fewer weights while achieving minimal accuracy loss. The prototype chip fabricated in 65-nm LP CMOS achieves 8.93/7.22 TOPS/W for 2-/3-layer LSTM RNNs trained with HCGS for TIMIT/TED-LIUM corpora.

KW - Hardware accelerator

KW - long short-term memory (LSTM)

KW - speech recognition

KW - structured sparsity weight compression

UR - http://www.scopus.com/inward/record.url?scp=85075908969&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075908969&partnerID=8YFLogxK

U2 - 10.1109/lssc.2019.2936761

DO - 10.1109/lssc.2019.2936761

M3 - Conference contribution

AN - SCOPUS:85075908969

T3 - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

SP - 119

EP - 122

BT - ESSCIRC 2019 - IEEE 45th European Solid State Circuits Conference

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 45th IEEE European Solid State Circuits Conference, ESSCIRC 2019

Y2 - 23 September 2019 through 26 September 2019

ER -

A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity with All Parameters Stored On-Chip

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this