Task-parallel analysis of molecular dynamics trajectories

Ioannis Paraskevakos, George Chantzialexiou, Andre Luckow, Thomas E. Cheatham, Mahzad Khoshlessan, Oliver Beckstein, Geoffrey C. Fox, Shantenu Jha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Different parallel frameworks for implementing data analysis applications have been proposed by the HPC and Big Data communities. In this paper, we investigate three task-parallel frameworks: Spark, Dask and RADICAL-Pilot with respect to their ability to support data analytics on HPC resources and compare them to MPI. We investigate the data analysis requirements of Molecular Dynamics (MD) simulations which are significant consumers of supercomputing cycles, producing immense amounts of data. A typical large-scale MD simulation of a physical system of O(100k) atoms over secs can produce from O(10) GB to O(1000) GBs of data. We propose and evaluate different approaches for parallelization of a representative set of MD trajectory analysis algorithms, in particular the computation of path similarity and leaflet identification. We evaluate Spark, Dask and RADICAL-Pilot with respect to their abstractions and runtime engine capabilities to support these algorithms. We provide a conceptual basis for comparing and understanding different frameworks that enable users to select the optimal system for each application. We also provide a quantitative performance analysis of the different algorithms across the three frameworks.

Original languageEnglish (US)
Title of host publicationProceedings of the 47th International Conference on Parallel Processing, ICPP 2018
PublisherAssociation for Computing Machinery
ISBN (Print)9781450365109
DOIs
StatePublished - Aug 13 2018
Event47th International Conference on Parallel Processing, ICPP 2018 - Eugene, United States
Duration: Aug 14 2018Aug 16 2018

Other

Other47th International Conference on Parallel Processing, ICPP 2018
CountryUnited States
CityEugene
Period8/14/188/16/18

Fingerprint

Molecular dynamics
Trajectories
Electric sparks
Optimal systems
Computer simulation
Engines
Atoms
Big data

Keywords

  • Data analytics
  • MD analysis
  • MD simulations analysis
  • Task-parallel

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Paraskevakos, I., Chantzialexiou, G., Luckow, A., Cheatham, T. E., Khoshlessan, M., Beckstein, O., ... Jha, S. (2018). Task-parallel analysis of molecular dynamics trajectories. In Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018 [a49] Association for Computing Machinery. https://doi.org/10.1145/3225058.3225128

Task-parallel analysis of molecular dynamics trajectories. / Paraskevakos, Ioannis; Chantzialexiou, George; Luckow, Andre; Cheatham, Thomas E.; Khoshlessan, Mahzad; Beckstein, Oliver; Fox, Geoffrey C.; Jha, Shantenu.

Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018. Association for Computing Machinery, 2018. a49.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Paraskevakos, I, Chantzialexiou, G, Luckow, A, Cheatham, TE, Khoshlessan, M, Beckstein, O, Fox, GC & Jha, S 2018, Task-parallel analysis of molecular dynamics trajectories. in Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018., a49, Association for Computing Machinery, 47th International Conference on Parallel Processing, ICPP 2018, Eugene, United States, 8/14/18. https://doi.org/10.1145/3225058.3225128
Paraskevakos I, Chantzialexiou G, Luckow A, Cheatham TE, Khoshlessan M, Beckstein O et al. Task-parallel analysis of molecular dynamics trajectories. In Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018. Association for Computing Machinery. 2018. a49 https://doi.org/10.1145/3225058.3225128
Paraskevakos, Ioannis ; Chantzialexiou, George ; Luckow, Andre ; Cheatham, Thomas E. ; Khoshlessan, Mahzad ; Beckstein, Oliver ; Fox, Geoffrey C. ; Jha, Shantenu. / Task-parallel analysis of molecular dynamics trajectories. Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018. Association for Computing Machinery, 2018.
@inproceedings{158c4d2f043543d69e8d0df592881a54,
title = "Task-parallel analysis of molecular dynamics trajectories",
abstract = "Different parallel frameworks for implementing data analysis applications have been proposed by the HPC and Big Data communities. In this paper, we investigate three task-parallel frameworks: Spark, Dask and RADICAL-Pilot with respect to their ability to support data analytics on HPC resources and compare them to MPI. We investigate the data analysis requirements of Molecular Dynamics (MD) simulations which are significant consumers of supercomputing cycles, producing immense amounts of data. A typical large-scale MD simulation of a physical system of O(100k) atoms over secs can produce from O(10) GB to O(1000) GBs of data. We propose and evaluate different approaches for parallelization of a representative set of MD trajectory analysis algorithms, in particular the computation of path similarity and leaflet identification. We evaluate Spark, Dask and RADICAL-Pilot with respect to their abstractions and runtime engine capabilities to support these algorithms. We provide a conceptual basis for comparing and understanding different frameworks that enable users to select the optimal system for each application. We also provide a quantitative performance analysis of the different algorithms across the three frameworks.",
keywords = "Data analytics, MD analysis, MD simulations analysis, Task-parallel",
author = "Ioannis Paraskevakos and George Chantzialexiou and Andre Luckow and Cheatham, {Thomas E.} and Mahzad Khoshlessan and Oliver Beckstein and Fox, {Geoffrey C.} and Shantenu Jha",
year = "2018",
month = "8",
day = "13",
doi = "10.1145/3225058.3225128",
language = "English (US)",
isbn = "9781450365109",
booktitle = "Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Task-parallel analysis of molecular dynamics trajectories

AU - Paraskevakos, Ioannis

AU - Chantzialexiou, George

AU - Luckow, Andre

AU - Cheatham, Thomas E.

AU - Khoshlessan, Mahzad

AU - Beckstein, Oliver

AU - Fox, Geoffrey C.

AU - Jha, Shantenu

PY - 2018/8/13

Y1 - 2018/8/13

N2 - Different parallel frameworks for implementing data analysis applications have been proposed by the HPC and Big Data communities. In this paper, we investigate three task-parallel frameworks: Spark, Dask and RADICAL-Pilot with respect to their ability to support data analytics on HPC resources and compare them to MPI. We investigate the data analysis requirements of Molecular Dynamics (MD) simulations which are significant consumers of supercomputing cycles, producing immense amounts of data. A typical large-scale MD simulation of a physical system of O(100k) atoms over secs can produce from O(10) GB to O(1000) GBs of data. We propose and evaluate different approaches for parallelization of a representative set of MD trajectory analysis algorithms, in particular the computation of path similarity and leaflet identification. We evaluate Spark, Dask and RADICAL-Pilot with respect to their abstractions and runtime engine capabilities to support these algorithms. We provide a conceptual basis for comparing and understanding different frameworks that enable users to select the optimal system for each application. We also provide a quantitative performance analysis of the different algorithms across the three frameworks.

AB - Different parallel frameworks for implementing data analysis applications have been proposed by the HPC and Big Data communities. In this paper, we investigate three task-parallel frameworks: Spark, Dask and RADICAL-Pilot with respect to their ability to support data analytics on HPC resources and compare them to MPI. We investigate the data analysis requirements of Molecular Dynamics (MD) simulations which are significant consumers of supercomputing cycles, producing immense amounts of data. A typical large-scale MD simulation of a physical system of O(100k) atoms over secs can produce from O(10) GB to O(1000) GBs of data. We propose and evaluate different approaches for parallelization of a representative set of MD trajectory analysis algorithms, in particular the computation of path similarity and leaflet identification. We evaluate Spark, Dask and RADICAL-Pilot with respect to their abstractions and runtime engine capabilities to support these algorithms. We provide a conceptual basis for comparing and understanding different frameworks that enable users to select the optimal system for each application. We also provide a quantitative performance analysis of the different algorithms across the three frameworks.

KW - Data analytics

KW - MD analysis

KW - MD simulations analysis

KW - Task-parallel

UR - http://www.scopus.com/inward/record.url?scp=85054786481&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054786481&partnerID=8YFLogxK

U2 - 10.1145/3225058.3225128

DO - 10.1145/3225058.3225128

M3 - Conference contribution

AN - SCOPUS:85054786481

SN - 9781450365109

BT - Proceedings of the 47th International Conference on Parallel Processing, ICPP 2018

PB - Association for Computing Machinery

ER -