A Latency-Optimized Reconfigurable NoC for In-Memory Acceleration of DNNs

Sumit K. Mandal, Gokul Krishnan, Chaitali Chakrabarti, Jae Sun Seo, Yu Cao, Umit Y. Ogras

Research output: Contribution to journalArticlepeer-review

Abstract

In-memory computing reduces latency and energy consumption of Deep Neural Networks (DNNs) by reducing the number of off-chip memory accesses. However, crossbar-based in-memory computing may significantly increase the volume of on-chip communication since the weights and activations are on-chip. State-of-the-art interconnect methodologies for in-memory computing deploy a bus-based network or mesh-based Network-on-Chip (NoC). Our experiments show that up to 90% of the total inference latency of a DNN hardware is spent on on-chip communication when the bus-based network is used. To reduce the communication latency, we propose a methodology to generate an NoC architecture along with a scheduling technique customized for different DNNs. We prove mathematically that the generated NoC architecture and corresponding schedules achieve the minimum possible communication latency for a given DNN. Furthermore, we generalize the proposed solution for edge computing and cloud computing. Experimental evaluations on a wide range of DNNs show that the proposed NoC architecture enables 20%-80% reduction in communication latency with respect to state-of-the-art interconnect solutions.

Original languageEnglish (US)
Article number9164917
Pages (from-to)362-375
Number of pages14
JournalIEEE Journal on Emerging and Selected Topics in Circuits and Systems
Volume10
Issue number3
DOIs
StatePublished - Sep 2020

Keywords

  • In-memory computing
  • deep neural networks
  • interconnect
  • network-on-chip
  • neural network accelerator

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'A Latency-Optimized Reconfigurable NoC for In-Memory Acceleration of DNNs'. Together they form a unique fingerprint.

Cite this