On the design and implementation of a simulator for parallel file system research

Yonggang Liu, Renato Figueiredo, Yiqi Xu, Ming Zhao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.

Original languageEnglish (US)
Title of host publicationIEEE Symposium on Mass Storage Systems and Technologies
DOIs
StatePublished - 2013
Externally publishedYes
Event2013 IEEE 29th Symposium on Mass Storage Systems and Technologies, MSST 2013 - Long Beach, CA, United States
Duration: May 6 2013May 10 2013

Other

Other2013 IEEE 29th Symposium on Mass Storage Systems and Technologies, MSST 2013
CountryUnited States
CityLong Beach, CA
Period5/6/135/10/13

Fingerprint

Simulators
Systems analysis
Servers
Experiments
Costs

Keywords

  • I/O scheduling
  • parallel file system
  • simulation

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Liu, Y., Figueiredo, R., Xu, Y., & Zhao, M. (2013). On the design and implementation of a simulator for parallel file system research. In IEEE Symposium on Mass Storage Systems and Technologies [6558438] https://doi.org/10.1109/MSST.2013.6558438

On the design and implementation of a simulator for parallel file system research. / Liu, Yonggang; Figueiredo, Renato; Xu, Yiqi; Zhao, Ming.

IEEE Symposium on Mass Storage Systems and Technologies. 2013. 6558438.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, Y, Figueiredo, R, Xu, Y & Zhao, M 2013, On the design and implementation of a simulator for parallel file system research. in IEEE Symposium on Mass Storage Systems and Technologies., 6558438, 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies, MSST 2013, Long Beach, CA, United States, 5/6/13. https://doi.org/10.1109/MSST.2013.6558438
Liu Y, Figueiredo R, Xu Y, Zhao M. On the design and implementation of a simulator for parallel file system research. In IEEE Symposium on Mass Storage Systems and Technologies. 2013. 6558438 https://doi.org/10.1109/MSST.2013.6558438
Liu, Yonggang ; Figueiredo, Renato ; Xu, Yiqi ; Zhao, Ming. / On the design and implementation of a simulator for parallel file system research. IEEE Symposium on Mass Storage Systems and Technologies. 2013.
@inproceedings{90e107bd6a2b41d3be1c7d71550d9564,
title = "On the design and implementation of a simulator for parallel file system research",
abstract = "Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.",
keywords = "I/O scheduling, parallel file system, simulation",
author = "Yonggang Liu and Renato Figueiredo and Yiqi Xu and Ming Zhao",
year = "2013",
doi = "10.1109/MSST.2013.6558438",
language = "English (US)",
isbn = "9781479902170",
booktitle = "IEEE Symposium on Mass Storage Systems and Technologies",

}

TY - GEN

T1 - On the design and implementation of a simulator for parallel file system research

AU - Liu, Yonggang

AU - Figueiredo, Renato

AU - Xu, Yiqi

AU - Zhao, Ming

PY - 2013

Y1 - 2013

N2 - Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.

AB - Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.

KW - I/O scheduling

KW - parallel file system

KW - simulation

UR - http://www.scopus.com/inward/record.url?scp=84883072746&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84883072746&partnerID=8YFLogxK

U2 - 10.1109/MSST.2013.6558438

DO - 10.1109/MSST.2013.6558438

M3 - Conference contribution

AN - SCOPUS:84883072746

SN - 9781479902170

BT - IEEE Symposium on Mass Storage Systems and Technologies

ER -