Power, performance, and area benefit of monolithic 3D ICs for on-chip deep neural networks targeting speech recognition

Kyungwook Chang, Deepak Kadetotad, Yu Cao, Jae-sun Seo, Sung Kyu Lim

Research output: Contribution to journalArticle

Abstract

In recent years, deep learning has become widespread for various real-world recognition tasks. In addition to recognition accuracy, energy efficiency and speed (i.e., performance) are other grand challenges to enable local intelligence in edge devices. In this article, we investigate the adoption of monolithic three-dimensional (3D) IC (M3D) technology for deep learning hardware design, using speech recognition as a test vehicle. M3D has recently proven to be one of the leading contenders to address the power, performance, and area (PPA) scaling challenges in advanced technology nodes. Our study encompasses the influence of key parameters in DNN hardware implementations towards their performance and energy efficiency, including DNN architectural choices, underlying workloads, and tier partitioning choices in M3D designs. Our post-layout M3D designs, together with hardware-efficient sparse algorithms, produce power savings and performance improvement beyond what can be achieved using conventional 2D ICs. Experimental results show that M3D offers 22.3% iso-performance power saving and 6.2% performance improvement, convincingly demonstrating its entitlement as a solution for DNN ASICs. We further present architectural and physical design guidelines for M3D DNNs to maximize the benefits.

Original languageEnglish (US)
Article numbera42
JournalACM Journal on Emerging Technologies in Computing Systems
Volume14
Issue number4
DOIs
StatePublished - Nov 1 2018

Fingerprint

Speech recognition
Hardware
Energy efficiency
Application specific integrated circuits
Deep neural networks
Deep learning

Keywords

  • High performance design
  • Low power design
  • Monolithic 3D IC
  • On-chip deep neural networks
  • Speech recognition

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Power, performance, and area benefit of monolithic 3D ICs for on-chip deep neural networks targeting speech recognition. / Chang, Kyungwook; Kadetotad, Deepak; Cao, Yu; Seo, Jae-sun; Kyu Lim, Sung.

In: ACM Journal on Emerging Technologies in Computing Systems, Vol. 14, No. 4, a42, 01.11.2018.

Research output: Contribution to journalArticle

@article{ba6be5f372fc4169adc9d64d111b9cc6,
title = "Power, performance, and area benefit of monolithic 3D ICs for on-chip deep neural networks targeting speech recognition",
abstract = "In recent years, deep learning has become widespread for various real-world recognition tasks. In addition to recognition accuracy, energy efficiency and speed (i.e., performance) are other grand challenges to enable local intelligence in edge devices. In this article, we investigate the adoption of monolithic three-dimensional (3D) IC (M3D) technology for deep learning hardware design, using speech recognition as a test vehicle. M3D has recently proven to be one of the leading contenders to address the power, performance, and area (PPA) scaling challenges in advanced technology nodes. Our study encompasses the influence of key parameters in DNN hardware implementations towards their performance and energy efficiency, including DNN architectural choices, underlying workloads, and tier partitioning choices in M3D designs. Our post-layout M3D designs, together with hardware-efficient sparse algorithms, produce power savings and performance improvement beyond what can be achieved using conventional 2D ICs. Experimental results show that M3D offers 22.3{\%} iso-performance power saving and 6.2{\%} performance improvement, convincingly demonstrating its entitlement as a solution for DNN ASICs. We further present architectural and physical design guidelines for M3D DNNs to maximize the benefits.",
keywords = "High performance design, Low power design, Monolithic 3D IC, On-chip deep neural networks, Speech recognition",
author = "Kyungwook Chang and Deepak Kadetotad and Yu Cao and Jae-sun Seo and {Kyu Lim}, Sung",
year = "2018",
month = "11",
day = "1",
doi = "10.1145/3273956",
language = "English (US)",
volume = "14",
journal = "ACM Journal on Emerging Technologies in Computing Systems",
issn = "1550-4832",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - Power, performance, and area benefit of monolithic 3D ICs for on-chip deep neural networks targeting speech recognition

AU - Chang, Kyungwook

AU - Kadetotad, Deepak

AU - Cao, Yu

AU - Seo, Jae-sun

AU - Kyu Lim, Sung

PY - 2018/11/1

Y1 - 2018/11/1

N2 - In recent years, deep learning has become widespread for various real-world recognition tasks. In addition to recognition accuracy, energy efficiency and speed (i.e., performance) are other grand challenges to enable local intelligence in edge devices. In this article, we investigate the adoption of monolithic three-dimensional (3D) IC (M3D) technology for deep learning hardware design, using speech recognition as a test vehicle. M3D has recently proven to be one of the leading contenders to address the power, performance, and area (PPA) scaling challenges in advanced technology nodes. Our study encompasses the influence of key parameters in DNN hardware implementations towards their performance and energy efficiency, including DNN architectural choices, underlying workloads, and tier partitioning choices in M3D designs. Our post-layout M3D designs, together with hardware-efficient sparse algorithms, produce power savings and performance improvement beyond what can be achieved using conventional 2D ICs. Experimental results show that M3D offers 22.3% iso-performance power saving and 6.2% performance improvement, convincingly demonstrating its entitlement as a solution for DNN ASICs. We further present architectural and physical design guidelines for M3D DNNs to maximize the benefits.

AB - In recent years, deep learning has become widespread for various real-world recognition tasks. In addition to recognition accuracy, energy efficiency and speed (i.e., performance) are other grand challenges to enable local intelligence in edge devices. In this article, we investigate the adoption of monolithic three-dimensional (3D) IC (M3D) technology for deep learning hardware design, using speech recognition as a test vehicle. M3D has recently proven to be one of the leading contenders to address the power, performance, and area (PPA) scaling challenges in advanced technology nodes. Our study encompasses the influence of key parameters in DNN hardware implementations towards their performance and energy efficiency, including DNN architectural choices, underlying workloads, and tier partitioning choices in M3D designs. Our post-layout M3D designs, together with hardware-efficient sparse algorithms, produce power savings and performance improvement beyond what can be achieved using conventional 2D ICs. Experimental results show that M3D offers 22.3% iso-performance power saving and 6.2% performance improvement, convincingly demonstrating its entitlement as a solution for DNN ASICs. We further present architectural and physical design guidelines for M3D DNNs to maximize the benefits.

KW - High performance design

KW - Low power design

KW - Monolithic 3D IC

KW - On-chip deep neural networks

KW - Speech recognition

UR - http://www.scopus.com/inward/record.url?scp=85057725846&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057725846&partnerID=8YFLogxK

U2 - 10.1145/3273956

DO - 10.1145/3273956

M3 - Article

VL - 14

JO - ACM Journal on Emerging Technologies in Computing Systems

JF - ACM Journal on Emerging Technologies in Computing Systems

SN - 1550-4832

IS - 4

M1 - a42

ER -