Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

Shihui Yin; Shreyas K. Venkataramanaiah; Gregory K. Chen; Ram Krishnamurthy; Yu Cao; Chaitali Chakrabarti; Jae-sun Seo

doi:10.1109/BIOCAS.2017.8325230

Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

Shihui Yin, Shreyas K. Venkataramanaiah, Gregory K. Chen, Ram Krishnamurthy, Yu Cao, Chaitali Chakrabarti, Jae-sun Seo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Scopus citations

Abstract

We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).

Original language	English (US)
Title of host publication	2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1-4
Number of pages	4
Volume	2018-January
ISBN (Electronic)	9781509058037
DOIs	https://doi.org/10.1109/BIOCAS.2017.8325230
State	Published - Mar 23 2018
Event	2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Torino, Italy Duration: Oct 19 2017 → Oct 21 2017

Other

Other	2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017
Country	Italy
City	Torino
Period	10/19/17 → 10/21/17

Keywords

back propagation
neuromorphic hardware
Spiking neural networks
straight-through estimator

ASJC Scopus subject areas

Biomedical Engineering
Electrical and Electronic Engineering
Instrumentation

Access to Document

10.1109/BIOCAS.2017.8325230

Fingerprint Dive into the research topics of 'Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations'. Together they form a unique fingerprint.

Cite this

Yin, S., Venkataramanaiah, S. K., Chen, G. K., Krishnamurthy, R., Cao, Y., Chakrabarti, C., & Seo, J. (2018). Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. In 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings (Vol. 2018-January, pp. 1-4). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIOCAS.2017.8325230

Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. / Yin, Shihui; Venkataramanaiah, Shreyas K.; Chen, Gregory K.; Krishnamurthy, Ram; Cao, Yu ; Chakrabarti, Chaitali ; Seo, Jae-sun.

2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 1-4.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yin, S, Venkataramanaiah, SK, Chen, GK, Krishnamurthy, R, Cao, Y , Chakrabarti, C & Seo, J 2018, Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. in 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 1-4, 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017, Torino, Italy, 10/19/17. https://doi.org/10.1109/BIOCAS.2017.8325230

Yin S, Venkataramanaiah SK, Chen GK, Krishnamurthy R, Cao Y , Chakrabarti C et al. Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. In 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1-4 https://doi.org/10.1109/BIOCAS.2017.8325230

Yin, Shihui ; Venkataramanaiah, Shreyas K. ; Chen, Gregory K. ; Krishnamurthy, Ram ; Cao, Yu ; Chakrabarti, Chaitali ; Seo, Jae-sun. / Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1-4

@inproceedings{9dc8ad9ba35d4b1ebe8ae9d29a100d38,

title = "Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations",

abstract = "We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).",

keywords = "back propagation, neuromorphic hardware, Spiking neural networks, straight-through estimator",

author = "Shihui Yin and Venkataramanaiah, {Shreyas K.} and Chen, {Gregory K.} and Ram Krishnamurthy and Yu Cao and Chaitali Chakrabarti and Jae-sun Seo",

year = "2018",

month = mar,

day = "23",

doi = "10.1109/BIOCAS.2017.8325230",

language = "English (US)",

volume = "2018-January",

pages = "1--4",

booktitle = "2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 ; Conference date: 19-10-2017 Through 21-10-2017",

}

TY - GEN

T1 - Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

AU - Yin, Shihui

AU - Venkataramanaiah, Shreyas K.

AU - Chen, Gregory K.

AU - Krishnamurthy, Ram

AU - Cao, Yu

AU - Chakrabarti, Chaitali

AU - Seo, Jae-sun

PY - 2018/3/23

Y1 - 2018/3/23

N2 - We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).

AB - We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).

KW - back propagation

KW - neuromorphic hardware

KW - Spiking neural networks

KW - straight-through estimator

UR - http://www.scopus.com/inward/record.url?scp=85049926656&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049926656&partnerID=8YFLogxK

U2 - 10.1109/BIOCAS.2017.8325230

DO - 10.1109/BIOCAS.2017.8325230

M3 - Conference contribution

AN - SCOPUS:85049926656

VL - 2018-January

SP - 1

EP - 4

BT - 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017

Y2 - 19 October 2017 through 21 October 2017

ER -