Cross-Stack Workload Characterization of Deep Recommendation Systems

Samuel Hsia, Udit Gupta, Mark Wilkening, Carole Jean Wu, Gu Yeon Wei, David Brooks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Deep learning based recommendation systems form the backbone of most personalized cloud services. Though the computer architecture community has recently started to take notice of deep recommendation inference, the resulting solutions have taken wildly different approaches-ranging from near memory processing to at-scale optimizations. To better design future hardware systems for deep recommendation inference, we must first systematically examine and characterize the underlying systems-level impact of design decisions across the different levels of the execution stack. In this paper, we characterize eight industry-representative deep recommendation models at three different levels of the execution stack: algorithms and software, systems platforms, and hardware microarchitectures. Through this cross-stack characterization, we first show that system deployment choices (i.e., CPUs or GPUs, batch size granularity) can give us up to 15x speedup. To better understand the bottlenecks for further optimization, we look at both software operator usage breakdown and CPU frontend and backend microarchitectural inefficiencies. Finally, we model the correlation between key algorithmic model architecture features and hardware bottlenecks, revealing the absence of a single dominant algorithmic component behind each hardware bottleneck.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE International Symposium on Workload Characterization, IISWC 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages157-168
Number of pages12
ISBN (Electronic)9781728176451
DOIs
StatePublished - Oct 2020
Externally publishedYes
Event16th IEEE International Symposium on Workload Characterization, IISWC 2020 - Virtual, Beijing, China
Duration: Oct 27 2020Oct 29 2020

Publication series

NameProceedings - 2020 IEEE International Symposium on Workload Characterization, IISWC 2020

Conference

Conference16th IEEE International Symposium on Workload Characterization, IISWC 2020
Country/TerritoryChina
CityVirtual, Beijing
Period10/27/2010/29/20

Keywords

  • GPU
  • computer architecture
  • deep learning
  • inference
  • machine learning
  • microarchitecture
  • recommender system

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Cross-Stack Workload Characterization of Deep Recommendation Systems'. Together they form a unique fingerprint.

Cite this