Abstract
We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).
| Original language | English (US) |
|---|---|
| Title of host publication | 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 1-4 |
| Number of pages | 4 |
| Volume | 2018-January |
| ISBN (Electronic) | 9781509058037 |
| DOIs | |
| State | Published - Mar 23 2018 |
| Event | 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Torino, Italy Duration: Oct 19 2017 → Oct 21 2017 |
Other
| Other | 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 |
|---|---|
| Country | Italy |
| City | Torino |
| Period | 10/19/17 → 10/21/17 |
Fingerprint
Keywords
- back propagation
- neuromorphic hardware
- Spiking neural networks
- straight-through estimator
ASJC Scopus subject areas
- Biomedical Engineering
- Electrical and Electronic Engineering
- Instrumentation
Cite this
Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. / Yin, Shihui; Venkataramanaiah, Shreyas K.; Chen, Gregory K.; Krishnamurthy, Ram; Cao, Yu; Chakrabarti, Chaitali; Seo, Jae-sun.
2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 1-4.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations
AU - Yin, Shihui
AU - Venkataramanaiah, Shreyas K.
AU - Chen, Gregory K.
AU - Krishnamurthy, Ram
AU - Cao, Yu
AU - Chakrabarti, Chaitali
AU - Seo, Jae-sun
PY - 2018/3/23
Y1 - 2018/3/23
N2 - We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).
AB - We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).
KW - back propagation
KW - neuromorphic hardware
KW - Spiking neural networks
KW - straight-through estimator
UR - http://www.scopus.com/inward/record.url?scp=85049926656&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85049926656&partnerID=8YFLogxK
U2 - 10.1109/BIOCAS.2017.8325230
DO - 10.1109/BIOCAS.2017.8325230
M3 - Conference contribution
VL - 2018-January
SP - 1
EP - 4
BT - 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
ER -