Foreseer: Workload-Aware Data Storage for MapReduce

Jia Zou; Juwei Shi; Tongping Liu; Zhao Cao; Chen Wang

doi:10.1109/ICDCS.2015.89

Foreseer: Workload-Aware Data Storage for MapReduce

Jia Zou, Juwei Shi, Tongping Liu, Zhao Cao, Chen Wang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

Original language	English (US)
Title of host publication	Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	746-747
Number of pages	2
ISBN (Electronic)	9781467372145
DOIs	https://doi.org/10.1109/ICDCS.2015.89
State	Published - Jul 22 2015
Externally published	Yes
Event	35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015 - Columbus, United States Duration: Jun 29 2015 → Jul 2 2015

Publication series

Name	Proceedings - International Conference on Distributed Computing Systems
Volume	2015-July

Other

Other	35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015
Country/Territory	United States
City	Columbus
Period	6/29/15 → 7/2/15

ASJC Scopus subject areas

Software
Hardware and Architecture
Computer Networks and Communications

Access to Document

10.1109/ICDCS.2015.89

Cite this

Zou, J., Shi, J., Liu, T., Cao, Z., & Wang, C. (2015). Foreseer: Workload-Aware Data Storage for MapReduce. In Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015 (pp. 746-747). Article 7164967 (Proceedings - International Conference on Distributed Computing Systems; Vol. 2015-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDCS.2015.89

Foreseer: Workload-Aware Data Storage for MapReduce. / Zou, Jia; Shi, Juwei; Liu, Tongping et al.
Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 746-747 7164967 (Proceedings - International Conference on Distributed Computing Systems; Vol. 2015-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zou, J, Shi, J, Liu, T, Cao, Z & Wang, C 2015, Foreseer: Workload-Aware Data Storage for MapReduce. in Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015., 7164967, Proceedings - International Conference on Distributed Computing Systems, vol. 2015-July, Institute of Electrical and Electronics Engineers Inc., pp. 746-747, 35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015, Columbus, United States, 6/29/15. https://doi.org/10.1109/ICDCS.2015.89

Zou J, Shi J, Liu T, Cao Z, Wang C. Foreseer: Workload-Aware Data Storage for MapReduce. In Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 746-747. 7164967. (Proceedings - International Conference on Distributed Computing Systems). doi: 10.1109/ICDCS.2015.89

@inproceedings{c782886960ae43f380959fcb990c6fc8,

title = "Foreseer: Workload-Aware Data Storage for MapReduce",

abstract = "Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.",

author = "Jia Zou and Juwei Shi and Tongping Liu and Zhao Cao and Chen Wang",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015 ; Conference date: 29-06-2015 Through 02-07-2015",

year = "2015",

month = jul,

day = "22",

doi = "10.1109/ICDCS.2015.89",

language = "English (US)",

series = "Proceedings - International Conference on Distributed Computing Systems",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "746--747",

booktitle = "Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015",

}

TY - GEN

T1 - Foreseer

T2 - 35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015

AU - Zou, Jia

AU - Shi, Juwei

AU - Liu, Tongping

AU - Cao, Zhao

AU - Wang, Chen

PY - 2015/7/22

Y1 - 2015/7/22

N2 - Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

AB - Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads' data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

UR - http://www.scopus.com/inward/record.url?scp=84944317372&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944317372&partnerID=8YFLogxK

U2 - 10.1109/ICDCS.2015.89

DO - 10.1109/ICDCS.2015.89

M3 - Conference contribution

AN - SCOPUS:84944317372

T3 - Proceedings - International Conference on Distributed Computing Systems

SP - 746

EP - 747

BT - Proceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 29 June 2015 through 2 July 2015

ER -

Foreseer: Workload-Aware Data Storage for MapReduce

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this