An architecture for lifecycle management in very large file systems

Akshat Verma, David Pease, Upendra Sharma, Marc Kaplan, Jim Rubas, Rohit Jain, Murthy Devarakonda, Mandis Beigi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations

Abstract

We present a policy-based architecture STEPS for life-cycle management (LCM) in a mass scale distributed file system. The STEPS architecture is designed in the context of IBM's SAN File System (SFS) and leverages the parallelism and scalability offered by SFS, while providing a centralized point of control for policy-based management. The architecture uses novel concepts like Policy Cache and Rate-Controlled Migration for efficient and non-intrusive execution of the LCM functions, while ensuring that the architecture scales with very large number of files. The architecture has been implemented and used for lifecycle management in a distributed deployment of SFS with heterogeneous data. We conduct experiments on the implementation to study the performance of the architecture. We observed that STEPS is highly scalable with increase in the number as well as the size of the file objects hosted by SFS. The performance study also demonstrated that most of the efficiency of policy execution is derived from Policy Cache. Further, a rate-control mechanism is necessary to ensure that users are isolated from LCM operations.

Original languageEnglish (US)
Title of host publicationProceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies
Pages160-168
Number of pages9
DOIs
StatePublished - 2005
Externally publishedYes
EventTwenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE/NASA MSST2005 - Monterey, CA, United States
Duration: Apr 11 2005Apr 14 2005

Publication series

NameProceedings - Twenty -second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies

Other

OtherTwenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE/NASA MSST2005
Country/TerritoryUnited States
CityMonterey, CA
Period4/11/054/14/05

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'An architecture for lifecycle management in very large file systems'. Together they form a unique fingerprint.

Cite this