Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

Shihui Yin, Shreyas K. Venkataramanaiah, Gregory K. Chen, Ram Krishnamurthy, Yu Cao, Chaitali Chakrabarti, Jae-sun Seo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 28nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (51.4-773 nJ/image).

Original languageEnglish (US)
Title of host publication2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-4
Number of pages4
Volume2018-January
ISBN (Electronic)9781509058037
DOIs
StatePublished - Mar 23 2018
Event2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Torino, Italy
Duration: Oct 19 2017Oct 21 2017

Other

Other2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017
CountryItaly
CityTorino
Period10/19/1710/21/17

Keywords

  • back propagation
  • neuromorphic hardware
  • Spiking neural networks
  • straight-through estimator

ASJC Scopus subject areas

  • Biomedical Engineering
  • Electrical and Electronic Engineering
  • Instrumentation

Fingerprint Dive into the research topics of 'Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations'. Together they form a unique fingerprint.

  • Cite this

    Yin, S., Venkataramanaiah, S. K., Chen, G. K., Krishnamurthy, R., Cao, Y., Chakrabarti, C., & Seo, J. (2018). Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. In 2017 IEEE Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceedings (Vol. 2018-January, pp. 1-4). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIOCAS.2017.8325230