Foreseer: Workload-Aware Data Storage for MapReduce

Jia Zou, Juwei Shi, Tongping Liu, Zhao Cao, Chen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages746-747
Number of pages2
ISBN (Electronic)9781467372145
DOIs
StatePublished - Jul 22 2015
Externally publishedYes
Event35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015 - Columbus, United States
Duration: Jun 29 2015Jul 2 2015

Publication series

NameProceedings - International Conference on Distributed Computing Systems
Volume2015-July

Other

Other35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015
CountryUnited States
CityColumbus
Period6/29/157/2/15

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Foreseer: Workload-Aware Data Storage for MapReduce'. Together they form a unique fingerprint.

  • Cite this

    Zou, J., Shi, J., Liu, T., Cao, Z., & Wang, C. (2015). Foreseer: Workload-Aware Data Storage for MapReduce. In Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015 (pp. 746-747). [7164967] (Proceedings - International Conference on Distributed Computing Systems; Vol. 2015-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDCS.2015.89