Towards efficient neural networks on-a-chip

Joint hardware-algorithm approaches

Xiaocong Du, Gokul Krishnan, Abinash Mohanty, Zheng Li, Gouranga Charan, Yu Cao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory volume, memory bandwidth and interconnection overhead. The adoption of the crossbar architecture with emerging memory technology partially solves the problem but induces process variation and other concerns. In this paper, we will present novel solutions to two fundamental issues in crossbar implementation of Artificial Intelligence (AI) algorithms: device variation and insufficient interconnections. These solutions are inspired by the statistical properties of algorithms themselves, especially the redundancy in neural network nodes and connections. By Random Sparse Adaptation and pruning the connections following the Small-World model, we demonstrate robust and efficient performance on representative datasets such as MNIST and CIFAR-10. Moreover, we present Continuous Growth and Pruning algorithm for future learning and adaptation on hardware.

Original languageEnglish (US)
Title of host publicationChina Semiconductor Technology International Conference 2019, CSTIC 2019
EditorsQinghuang Lin, Kafai Lai, Cor Claeys, Steve Liang, Peilin Song, Wenjian Yu, Hanming Wu, Xinping Qu, Hsiang-Lan Lung, Ru Huang, Zhen Guo, Ying Zhang
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538674437
DOIs
StatePublished - Mar 1 2019
Event2019 China Semiconductor Technology International Conference, CSTIC 2019 - Shanghai, China
Duration: Mar 18 2019Mar 19 2019

Publication series

NameChina Semiconductor Technology International Conference 2019, CSTIC 2019

Conference

Conference2019 China Semiconductor Technology International Conference, CSTIC 2019
CountryChina
CityShanghai
Period3/18/193/19/19

Fingerprint

hardware
chips
Neural networks
Hardware
Data storage equipment
artificial intelligence
Learning algorithms
machine learning
Artificial intelligence
Redundancy
Learning systems
redundancy
learning
Bandwidth
emerging
platforms
bandwidth

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Instrumentation

Cite this

Du, X., Krishnan, G., Mohanty, A., Li, Z., Charan, G., & Cao, Y. (2019). Towards efficient neural networks on-a-chip: Joint hardware-algorithm approaches. In Q. Lin, K. Lai, C. Claeys, S. Liang, P. Song, W. Yu, H. Wu, X. Qu, H-L. Lung, R. Huang, Z. Guo, ... Y. Zhang (Eds.), China Semiconductor Technology International Conference 2019, CSTIC 2019 [8755608] (China Semiconductor Technology International Conference 2019, CSTIC 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CSTIC.2019.8755608

Towards efficient neural networks on-a-chip : Joint hardware-algorithm approaches. / Du, Xiaocong; Krishnan, Gokul; Mohanty, Abinash; Li, Zheng; Charan, Gouranga; Cao, Yu.

China Semiconductor Technology International Conference 2019, CSTIC 2019. ed. / Qinghuang Lin; Kafai Lai; Cor Claeys; Steve Liang; Peilin Song; Wenjian Yu; Hanming Wu; Xinping Qu; Hsiang-Lan Lung; Ru Huang; Zhen Guo; Ying Zhang. Institute of Electrical and Electronics Engineers Inc., 2019. 8755608 (China Semiconductor Technology International Conference 2019, CSTIC 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Du, X, Krishnan, G, Mohanty, A, Li, Z, Charan, G & Cao, Y 2019, Towards efficient neural networks on-a-chip: Joint hardware-algorithm approaches. in Q Lin, K Lai, C Claeys, S Liang, P Song, W Yu, H Wu, X Qu, H-L Lung, R Huang, Z Guo & Y Zhang (eds), China Semiconductor Technology International Conference 2019, CSTIC 2019., 8755608, China Semiconductor Technology International Conference 2019, CSTIC 2019, Institute of Electrical and Electronics Engineers Inc., 2019 China Semiconductor Technology International Conference, CSTIC 2019, Shanghai, China, 3/18/19. https://doi.org/10.1109/CSTIC.2019.8755608
Du X, Krishnan G, Mohanty A, Li Z, Charan G, Cao Y. Towards efficient neural networks on-a-chip: Joint hardware-algorithm approaches. In Lin Q, Lai K, Claeys C, Liang S, Song P, Yu W, Wu H, Qu X, Lung H-L, Huang R, Guo Z, Zhang Y, editors, China Semiconductor Technology International Conference 2019, CSTIC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8755608. (China Semiconductor Technology International Conference 2019, CSTIC 2019). https://doi.org/10.1109/CSTIC.2019.8755608
Du, Xiaocong ; Krishnan, Gokul ; Mohanty, Abinash ; Li, Zheng ; Charan, Gouranga ; Cao, Yu. / Towards efficient neural networks on-a-chip : Joint hardware-algorithm approaches. China Semiconductor Technology International Conference 2019, CSTIC 2019. editor / Qinghuang Lin ; Kafai Lai ; Cor Claeys ; Steve Liang ; Peilin Song ; Wenjian Yu ; Hanming Wu ; Xinping Qu ; Hsiang-Lan Lung ; Ru Huang ; Zhen Guo ; Ying Zhang. Institute of Electrical and Electronics Engineers Inc., 2019. (China Semiconductor Technology International Conference 2019, CSTIC 2019).
@inproceedings{7cf0171a44734baba6ff47ece37baac7,
title = "Towards efficient neural networks on-a-chip: Joint hardware-algorithm approaches",
abstract = "Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory volume, memory bandwidth and interconnection overhead. The adoption of the crossbar architecture with emerging memory technology partially solves the problem but induces process variation and other concerns. In this paper, we will present novel solutions to two fundamental issues in crossbar implementation of Artificial Intelligence (AI) algorithms: device variation and insufficient interconnections. These solutions are inspired by the statistical properties of algorithms themselves, especially the redundancy in neural network nodes and connections. By Random Sparse Adaptation and pruning the connections following the Small-World model, we demonstrate robust and efficient performance on representative datasets such as MNIST and CIFAR-10. Moreover, we present Continuous Growth and Pruning algorithm for future learning and adaptation on hardware.",
author = "Xiaocong Du and Gokul Krishnan and Abinash Mohanty and Zheng Li and Gouranga Charan and Yu Cao",
year = "2019",
month = "3",
day = "1",
doi = "10.1109/CSTIC.2019.8755608",
language = "English (US)",
series = "China Semiconductor Technology International Conference 2019, CSTIC 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
editor = "Qinghuang Lin and Kafai Lai and Cor Claeys and Steve Liang and Peilin Song and Wenjian Yu and Hanming Wu and Xinping Qu and Hsiang-Lan Lung and Ru Huang and Zhen Guo and Ying Zhang",
booktitle = "China Semiconductor Technology International Conference 2019, CSTIC 2019",

}

TY - GEN

T1 - Towards efficient neural networks on-a-chip

T2 - Joint hardware-algorithm approaches

AU - Du, Xiaocong

AU - Krishnan, Gokul

AU - Mohanty, Abinash

AU - Li, Zheng

AU - Charan, Gouranga

AU - Cao, Yu

PY - 2019/3/1

Y1 - 2019/3/1

N2 - Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory volume, memory bandwidth and interconnection overhead. The adoption of the crossbar architecture with emerging memory technology partially solves the problem but induces process variation and other concerns. In this paper, we will present novel solutions to two fundamental issues in crossbar implementation of Artificial Intelligence (AI) algorithms: device variation and insufficient interconnections. These solutions are inspired by the statistical properties of algorithms themselves, especially the redundancy in neural network nodes and connections. By Random Sparse Adaptation and pruning the connections following the Small-World model, we demonstrate robust and efficient performance on representative datasets such as MNIST and CIFAR-10. Moreover, we present Continuous Growth and Pruning algorithm for future learning and adaptation on hardware.

AB - Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory volume, memory bandwidth and interconnection overhead. The adoption of the crossbar architecture with emerging memory technology partially solves the problem but induces process variation and other concerns. In this paper, we will present novel solutions to two fundamental issues in crossbar implementation of Artificial Intelligence (AI) algorithms: device variation and insufficient interconnections. These solutions are inspired by the statistical properties of algorithms themselves, especially the redundancy in neural network nodes and connections. By Random Sparse Adaptation and pruning the connections following the Small-World model, we demonstrate robust and efficient performance on representative datasets such as MNIST and CIFAR-10. Moreover, we present Continuous Growth and Pruning algorithm for future learning and adaptation on hardware.

UR - http://www.scopus.com/inward/record.url?scp=85069499030&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069499030&partnerID=8YFLogxK

U2 - 10.1109/CSTIC.2019.8755608

DO - 10.1109/CSTIC.2019.8755608

M3 - Conference contribution

T3 - China Semiconductor Technology International Conference 2019, CSTIC 2019

BT - China Semiconductor Technology International Conference 2019, CSTIC 2019

A2 - Lin, Qinghuang

A2 - Lai, Kafai

A2 - Claeys, Cor

A2 - Liang, Steve

A2 - Song, Peilin

A2 - Yu, Wenjian

A2 - Wu, Hanming

A2 - Qu, Xinping

A2 - Lung, Hsiang-Lan

A2 - Huang, Ru

A2 - Guo, Zhen

A2 - Zhang, Ying

PB - Institute of Electrical and Electronics Engineers Inc.

ER -