A 2.6 TOPS/W 16-Bit Fixed-Point Convolutional Neural Network Learning Processor in 65-nm CMOS

Shihui Yin; Jae Sun Seo

doi:10.1109/LSSC.2019.2954780

A 2.6 TOPS/W 16-Bit Fixed-Point Convolutional Neural Network Learning Processor in 65-nm CMOS

Shihui Yin, Jae Sun Seo

Engineering, Ira A. Fulton Schools of (IAFSE)

Research output: Contribution to journal › Article › peer-review

17 Scopus citations

Abstract

We present a convolutional neural network (CNN) learning processor, which accelerates the stochastic gradient descent (SGD) with a momentum-based training algorithm in 16-bit fixed-point precision. Using a new cyclic weight storage and access scheme, we use the same off-the-shelf SRAMs for nontranspose and transpose operations during feedforward (FF) and feedbackward (FB) phases, respectively, of the CNN learning process. The 65-nm CNN learning processor achieves peak energy efficiency of 2.6 TOPS/W for 16-bit fixed-point operations, consuming 10.45 mW at 0.55 V.

Original language	English (US)
Article number	8907458
Pages (from-to)	13-16
Number of pages	4
Journal	IEEE Solid-State Circuits Letters
Volume	3
Issue number	1
DOIs	https://doi.org/10.1109/LSSC.2019.2954780
State	Published - Dec 2020

Keywords

Convolutional neural networks (CNNs)
dual-read-mode weight storage
on-chip learning
stochastic gradient descent (SGD)

ASJC Scopus subject areas

Electrical and Electronic Engineering

Access to Document

10.1109/LSSC.2019.2954780

Cite this

@article{12b6035344cf46eda4c16dbcea468b9c,

title = "A 2.6 TOPS/W 16-Bit Fixed-Point Convolutional Neural Network Learning Processor in 65-nm CMOS",

abstract = "We present a convolutional neural network (CNN) learning processor, which accelerates the stochastic gradient descent (SGD) with a momentum-based training algorithm in 16-bit fixed-point precision. Using a new cyclic weight storage and access scheme, we use the same off-the-shelf SRAMs for nontranspose and transpose operations during feedforward (FF) and feedbackward (FB) phases, respectively, of the CNN learning process. The 65-nm CNN learning processor achieves peak energy efficiency of 2.6 TOPS/W for 16-bit fixed-point operations, consuming 10.45 mW at 0.55 V.",

keywords = "Convolutional neural networks (CNNs), dual-read-mode weight storage, on-chip learning, stochastic gradient descent (SGD)",

author = "Shihui Yin and Seo, {Jae Sun}",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.",

year = "2020",

month = dec,

doi = "10.1109/LSSC.2019.2954780",

language = "English (US)",

volume = "3",

pages = "13--16",

journal = "IEEE Solid-State Circuits Letters",

issn = "2573-9603",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "1",

}

TY - JOUR

T1 - A 2.6 TOPS/W 16-Bit Fixed-Point Convolutional Neural Network Learning Processor in 65-nm CMOS

AU - Yin, Shihui

AU - Seo, Jae Sun

PY - 2020/12

Y1 - 2020/12

N2 - We present a convolutional neural network (CNN) learning processor, which accelerates the stochastic gradient descent (SGD) with a momentum-based training algorithm in 16-bit fixed-point precision. Using a new cyclic weight storage and access scheme, we use the same off-the-shelf SRAMs for nontranspose and transpose operations during feedforward (FF) and feedbackward (FB) phases, respectively, of the CNN learning process. The 65-nm CNN learning processor achieves peak energy efficiency of 2.6 TOPS/W for 16-bit fixed-point operations, consuming 10.45 mW at 0.55 V.

AB - We present a convolutional neural network (CNN) learning processor, which accelerates the stochastic gradient descent (SGD) with a momentum-based training algorithm in 16-bit fixed-point precision. Using a new cyclic weight storage and access scheme, we use the same off-the-shelf SRAMs for nontranspose and transpose operations during feedforward (FF) and feedbackward (FB) phases, respectively, of the CNN learning process. The 65-nm CNN learning processor achieves peak energy efficiency of 2.6 TOPS/W for 16-bit fixed-point operations, consuming 10.45 mW at 0.55 V.

KW - Convolutional neural networks (CNNs)

KW - dual-read-mode weight storage

KW - on-chip learning

KW - stochastic gradient descent (SGD)

UR - http://www.scopus.com/inward/record.url?scp=85082546739&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85082546739&partnerID=8YFLogxK

U2 - 10.1109/LSSC.2019.2954780

DO - 10.1109/LSSC.2019.2954780

M3 - Article

AN - SCOPUS:85082546739

SN - 2573-9603

VL - 3

SP - 13

EP - 16

JO - IEEE Solid-State Circuits Letters

JF - IEEE Solid-State Circuits Letters

IS - 1

M1 - 8907458

ER -

A 2.6 TOPS/W 16-Bit Fixed-Point Convolutional Neural Network Learning Processor in 65-nm CMOS

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this