An architecture for lifecycle management in very large file systems

Akshat Verma, David Pease, Upendra Sharma, Marc Kaplan, Jim Rubas, Rohit Jain, Murthy Devarakonda, Mandis Beigi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

We present a policy-based architecture STEPS for life-cycle management (LCM) in a mass scale distributed file system. The STEPS architecture is designed in the context of IBM's SAN File System (SFS) and leverages the parallelism and scalability offered by SFS, while providing a centralized point of control for policy-based management. The architecture uses novel concepts like Policy Cache and Rate-Controlled Migration for efficient and non-intrusive execution of the LCM functions, while ensuring that the architecture scales with very large number of files. The architecture has been implemented and used for lifecycle management in a distributed deployment of SFS with heterogeneous data. We conduct experiments on the implementation to study the performance of the architecture. We observed that STEPS is highly scalable with increase in the number as well as the size of the file objects hosted by SFS. The performance study also demonstrated that most of the efficiency of policy execution is derived from Policy Cache. Further, a rate-control mechanism is necessary to ensure that users are isolated from LCM operations.

Original languageEnglish (US)
Title of host publicationProceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies
Pages160-168
Number of pages9
DOIs
StatePublished - Oct 31 2005
Externally publishedYes
EventTwenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE/NASA MSST2005 - Monterey, CA, United States
Duration: Apr 11 2005Apr 14 2005

Other

OtherTwenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE/NASA MSST2005
CountryUnited States
CityMonterey, CA
Period4/11/054/14/05

Fingerprint

Life cycle
Scalability
Experiments

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Verma, A., Pease, D., Sharma, U., Kaplan, M., Rubas, J., Jain, R., ... Beigi, M. (2005). An architecture for lifecycle management in very large file systems. In Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies (pp. 160-168) https://doi.org/10.1109/MSST.2005.4

An architecture for lifecycle management in very large file systems. / Verma, Akshat; Pease, David; Sharma, Upendra; Kaplan, Marc; Rubas, Jim; Jain, Rohit; Devarakonda, Murthy; Beigi, Mandis.

Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies. 2005. p. 160-168.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Verma, A, Pease, D, Sharma, U, Kaplan, M, Rubas, J, Jain, R, Devarakonda, M & Beigi, M 2005, An architecture for lifecycle management in very large file systems. in Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies. pp. 160-168, Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE/NASA MSST2005, Monterey, CA, United States, 4/11/05. https://doi.org/10.1109/MSST.2005.4
Verma A, Pease D, Sharma U, Kaplan M, Rubas J, Jain R et al. An architecture for lifecycle management in very large file systems. In Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies. 2005. p. 160-168 https://doi.org/10.1109/MSST.2005.4
Verma, Akshat ; Pease, David ; Sharma, Upendra ; Kaplan, Marc ; Rubas, Jim ; Jain, Rohit ; Devarakonda, Murthy ; Beigi, Mandis. / An architecture for lifecycle management in very large file systems. Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies. 2005. pp. 160-168
@inproceedings{4c712f812db040ecac51417b44c43c64,
title = "An architecture for lifecycle management in very large file systems",
abstract = "We present a policy-based architecture STEPS for life-cycle management (LCM) in a mass scale distributed file system. The STEPS architecture is designed in the context of IBM's SAN File System (SFS) and leverages the parallelism and scalability offered by SFS, while providing a centralized point of control for policy-based management. The architecture uses novel concepts like Policy Cache and Rate-Controlled Migration for efficient and non-intrusive execution of the LCM functions, while ensuring that the architecture scales with very large number of files. The architecture has been implemented and used for lifecycle management in a distributed deployment of SFS with heterogeneous data. We conduct experiments on the implementation to study the performance of the architecture. We observed that STEPS is highly scalable with increase in the number as well as the size of the file objects hosted by SFS. The performance study also demonstrated that most of the efficiency of policy execution is derived from Policy Cache. Further, a rate-control mechanism is necessary to ensure that users are isolated from LCM operations.",
author = "Akshat Verma and David Pease and Upendra Sharma and Marc Kaplan and Jim Rubas and Rohit Jain and Murthy Devarakonda and Mandis Beigi",
year = "2005",
month = "10",
day = "31",
doi = "10.1109/MSST.2005.4",
language = "English (US)",
isbn = "0769523188",
pages = "160--168",
booktitle = "Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies",

}

TY - GEN

T1 - An architecture for lifecycle management in very large file systems

AU - Verma, Akshat

AU - Pease, David

AU - Sharma, Upendra

AU - Kaplan, Marc

AU - Rubas, Jim

AU - Jain, Rohit

AU - Devarakonda, Murthy

AU - Beigi, Mandis

PY - 2005/10/31

Y1 - 2005/10/31

N2 - We present a policy-based architecture STEPS for life-cycle management (LCM) in a mass scale distributed file system. The STEPS architecture is designed in the context of IBM's SAN File System (SFS) and leverages the parallelism and scalability offered by SFS, while providing a centralized point of control for policy-based management. The architecture uses novel concepts like Policy Cache and Rate-Controlled Migration for efficient and non-intrusive execution of the LCM functions, while ensuring that the architecture scales with very large number of files. The architecture has been implemented and used for lifecycle management in a distributed deployment of SFS with heterogeneous data. We conduct experiments on the implementation to study the performance of the architecture. We observed that STEPS is highly scalable with increase in the number as well as the size of the file objects hosted by SFS. The performance study also demonstrated that most of the efficiency of policy execution is derived from Policy Cache. Further, a rate-control mechanism is necessary to ensure that users are isolated from LCM operations.

AB - We present a policy-based architecture STEPS for life-cycle management (LCM) in a mass scale distributed file system. The STEPS architecture is designed in the context of IBM's SAN File System (SFS) and leverages the parallelism and scalability offered by SFS, while providing a centralized point of control for policy-based management. The architecture uses novel concepts like Policy Cache and Rate-Controlled Migration for efficient and non-intrusive execution of the LCM functions, while ensuring that the architecture scales with very large number of files. The architecture has been implemented and used for lifecycle management in a distributed deployment of SFS with heterogeneous data. We conduct experiments on the implementation to study the performance of the architecture. We observed that STEPS is highly scalable with increase in the number as well as the size of the file objects hosted by SFS. The performance study also demonstrated that most of the efficiency of policy execution is derived from Policy Cache. Further, a rate-control mechanism is necessary to ensure that users are isolated from LCM operations.

UR - http://www.scopus.com/inward/record.url?scp=27144504168&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=27144504168&partnerID=8YFLogxK

U2 - 10.1109/MSST.2005.4

DO - 10.1109/MSST.2005.4

M3 - Conference contribution

SN - 0769523188

SN - 9780769523187

SP - 160

EP - 168

BT - Proceedings - Twenty-second IEEE/Thirteenth NASA Goddard Conference on Mass Storage Systems and Technologies

ER -