vPFS: Bandwidth virtualization of parallel storage systems

Yiqi Xu, Dulcardo Arteaga, Ming Zhao, Yonggang Liu, Renato Figueiredo, Seetharami Seelam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

Existing parallel file systems are unable to differentiate I/Os requests from concurrent applications and meet per-application bandwidth requirements. This limitation prevents applications from meeting their desired Quality of Service (QoS) as high-performance computing (HPC) systems continue to scale up. This paper presents vPFS, a new solution to address this challenge through a bandwidth virtualization layer for parallel file systems. vPFS employs user-level parallel file system proxies to interpose requests between native clients and servers and to schedule parallel I/Os from different applications based on configurable bandwidth management policies. vPFS is designed to be generic enough to support various scheduling algorithms and parallel file systems. Its utility and performance are studied with a prototype which virtualizes PVFS2, a widely used parallel file system. Enhanced proportional sharing schedulers are enabled based on the unique characteristics (parallel striped I/Os) and requirement (high throughput) of parallel storage systems. The enhancements include new threshold- and layout-driven scheduling synchronization schemes which reduce global communication overhead while delivering total-service fairness. An experimental evaluation using typical HPC benchmarks (IOR, NPB BTIO) shows that the throughput overhead of vPFS is small (<3% for write, <1% for read). It also shows that vPFS can achieve good proportional bandwidth sharing (<96% of target sharing ratio) for competing applications with diverse I/O patterns.

Original languageEnglish (US)
Title of host publication2012 IEEE 28th Symposium on Mass Storage Systems and Technologies, MSST 2012
DOIs
StatePublished - Sep 18 2012
Externally publishedYes
Event2012 IEEE 28th Symposium on Mass Storage Systems and Technologies, MSST 2012 - Pacific Grove, CA, United States
Duration: Apr 16 2012Apr 20 2012

Publication series

NameIEEE Symposium on Mass Storage Systems and Technologies
ISSN (Print)2160-1968

Other

Other2012 IEEE 28th Symposium on Mass Storage Systems and Technologies, MSST 2012
CountryUnited States
CityPacific Grove, CA
Period4/16/124/20/12

Keywords

  • QoS
  • parallel storage
  • performance virtualization

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'vPFS: Bandwidth virtualization of parallel storage systems'. Together they form a unique fingerprint.

Cite this