TY - GEN
T1 - Emerging Neural Workloads and Their Impact on Hardware
AU - Brooks, David
AU - Frank, Martin M.
AU - Gokmen, Tayfun
AU - Gupta, Udit
AU - Hu, X. Sharon
AU - Jain, Shubham
AU - Laguna, Ann Franchesca
AU - Niemier, Michael
AU - O'Connor, Ian
AU - Raghunathan, Anand
AU - Ranjan, Ashish
AU - Reis, Dayane
AU - Stevens, Jacob R.
AU - Wu, Carole Jean
AU - Yin, Xunzhao
N1 - Funding Information:
ACKNOWLEDGMENTS This work was supported in part by ADA, ASCENT, and C-BRIC – three of the six centers in JUMP, a Semiconductor Research Corporation (SRC) program sponsored by DARPA.
Publisher Copyright:
© 2020 EDAA.
PY - 2020/3
Y1 - 2020/3
N2 - We consider existing and emerging neural workloads, and what hardware accelerators might be best suited for said workloads. We begin with a discussion of analog crossbar arrays, which are known to be well-suited for matrix-vector multiplication operations that are commonplace in existing neural network models such as convolutional neural networks (CNNs). We highlight candidate crosspoint devices, what device and materials challenges must be overcome for a given device to be employed in a crossbar array for a computationally interesting neural workload, and how circuit and algorithmic optimizations may be employed to mitigate undesirable characteristics from devices/materials. We then discuss two emerging neural workloads. We first consider machine learning models for one- and few-shot learning tasks (i.e., where a network can be trained with just one or a few, representative examples of a given class). Notably crossbar-based architectures can be used to accelerate said models. Hardware solutions based on content addressable memory arrays will also be discussed. We then consider machine learning models for recommendation systems. Recommendation models, an emerging class of machine learning models, employ distinct neural network architectures that operate of continuous and categorical input features which make hardware acceleration challenging. We will discuss the open research challenges and opportunities within this space.
AB - We consider existing and emerging neural workloads, and what hardware accelerators might be best suited for said workloads. We begin with a discussion of analog crossbar arrays, which are known to be well-suited for matrix-vector multiplication operations that are commonplace in existing neural network models such as convolutional neural networks (CNNs). We highlight candidate crosspoint devices, what device and materials challenges must be overcome for a given device to be employed in a crossbar array for a computationally interesting neural workload, and how circuit and algorithmic optimizations may be employed to mitigate undesirable characteristics from devices/materials. We then discuss two emerging neural workloads. We first consider machine learning models for one- and few-shot learning tasks (i.e., where a network can be trained with just one or a few, representative examples of a given class). Notably crossbar-based architectures can be used to accelerate said models. Hardware solutions based on content addressable memory arrays will also be discussed. We then consider machine learning models for recommendation systems. Recommendation models, an emerging class of machine learning models, employ distinct neural network architectures that operate of continuous and categorical input features which make hardware acceleration challenging. We will discuss the open research challenges and opportunities within this space.
UR - http://www.scopus.com/inward/record.url?scp=85087395829&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087395829&partnerID=8YFLogxK
U2 - 10.23919/DATE48585.2020.9116435
DO - 10.23919/DATE48585.2020.9116435
M3 - Conference contribution
AN - SCOPUS:85087395829
T3 - Proceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
SP - 1462
EP - 1471
BT - Proceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
A2 - Di Natale, Giorgio
A2 - Bolchini, Cristiana
A2 - Vatajelu, Elena-Ioana
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
Y2 - 9 March 2020 through 13 March 2020
ER -