Towards simulation of parallel file system scheduling algorithms with PFSsim

Yonggang Liu, Renato Figueiredo, Dulcardo Clavijo, Yiqi Xu, Ming Zhao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

26 Citations (Scopus)

Abstract

Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed (e.g., [1,2]), their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ [23] and the disk system simulator DiskSim [21]. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms.

Original languageEnglish (US)
Title of host publication2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011
StatePublished - 2011
Externally publishedYes
Event2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011 - Denver, CO, United States
Duration: May 25 2011May 25 2011

Other

Other2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011
CountryUnited States
CityDenver, CO
Period5/25/115/25/11

Fingerprint

Scheduling algorithms
Simulators
Scheduling

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Liu, Y., Figueiredo, R., Clavijo, D., Xu, Y., & Zhao, M. (2011). Towards simulation of parallel file system scheduling algorithms with PFSsim. In 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011

Towards simulation of parallel file system scheduling algorithms with PFSsim. / Liu, Yonggang; Figueiredo, Renato; Clavijo, Dulcardo; Xu, Yiqi; Zhao, Ming.

2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011. 2011.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, Y, Figueiredo, R, Clavijo, D, Xu, Y & Zhao, M 2011, Towards simulation of parallel file system scheduling algorithms with PFSsim. in 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011. 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011, Denver, CO, United States, 5/25/11.
Liu Y, Figueiredo R, Clavijo D, Xu Y, Zhao M. Towards simulation of parallel file system scheduling algorithms with PFSsim. In 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011. 2011
Liu, Yonggang ; Figueiredo, Renato ; Clavijo, Dulcardo ; Xu, Yiqi ; Zhao, Ming. / Towards simulation of parallel file system scheduling algorithms with PFSsim. 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011. 2011.
@inproceedings{950a1a8c8fef447d9b18a0c57e7ba193,
title = "Towards simulation of parallel file system scheduling algorithms with PFSsim",
abstract = "Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed (e.g., [1,2]), their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ [23] and the disk system simulator DiskSim [21]. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms.",
author = "Yonggang Liu and Renato Figueiredo and Dulcardo Clavijo and Yiqi Xu and Ming Zhao",
year = "2011",
language = "English (US)",
isbn = "9781457704284",
booktitle = "2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011",

}

TY - GEN

T1 - Towards simulation of parallel file system scheduling algorithms with PFSsim

AU - Liu, Yonggang

AU - Figueiredo, Renato

AU - Clavijo, Dulcardo

AU - Xu, Yiqi

AU - Zhao, Ming

PY - 2011

Y1 - 2011

N2 - Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed (e.g., [1,2]), their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ [23] and the disk system simulator DiskSim [21]. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms.

AB - Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed (e.g., [1,2]), their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ [23] and the disk system simulator DiskSim [21]. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms.

UR - http://www.scopus.com/inward/record.url?scp=84892707923&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84892707923&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781457704284

BT - 2011 IEEE 7th International Workshop on Storage Network Architecture and Parallel I/Os, SNAPI 2011

ER -