TY - JOUR
T1 - Efficient execution of the WRF model and other HPC applications in the cloud
AU - Duran-Limon, Hector A.
AU - Flores-Contreras, Jesus
AU - Parlavantzas, Nikos
AU - Zhao, Ming
AU - Meulenert-Peña, Angel
N1 - Publisher Copyright:
© 2016, Springer-Verlag Berlin Heidelberg.
PY - 2016/9/1
Y1 - 2016/9/1
N2 - There are many scientific applications that have high performance computing (HPC) demands. Such demands are traditionally supported by cluster- or Grid-based systems. Cloud computing, which has experienced a tremendous growth, emerged as an approach to provide on-demand access to computing resources. The cloud computing paradigm offers a number of advantages over other distributed platforms. For example, the access to resources is flexible and cost-effective since it is not necessary to invest a large amount of money on a computing infrastructure nor pay salaries for maintenance functions. Therefore, the possibility of using cloud computing for running high performance computing applications is attractive. However, it has been shown elsewhere that current cloud computing platforms are not suitable for running some of these kinds of applications since the performance offered is very poor. The reason is mainly the overhead from virtualisation which is extensively used by most cloud computing platforms as a means to optimise resource usage. Furthermore, running HPC applications in current cloud platforms is a complex task that in many cases requires configuring a cluster of virtual machines (VMs). In this paper, we present a lightweight virtualisation approach for efficiently running the Weather Research and Forecasting (WRF) model (a computing- and communication-intensive application) in a cloud computing environment. Our approach also provides a higher-level programming model that automates the process of configuring a cluster of VMs. We assume such a cloud environment can be shared with other types of HPC applications such as mpiBLAST (an embarrassingly parallel application), and MiniFE (a memory-intensive application). Our experimental results show that lightweight virtualisation imposes about 5 % overhead and it substantially outperforms traditional heavyweight virtualisation such as KVM.
AB - There are many scientific applications that have high performance computing (HPC) demands. Such demands are traditionally supported by cluster- or Grid-based systems. Cloud computing, which has experienced a tremendous growth, emerged as an approach to provide on-demand access to computing resources. The cloud computing paradigm offers a number of advantages over other distributed platforms. For example, the access to resources is flexible and cost-effective since it is not necessary to invest a large amount of money on a computing infrastructure nor pay salaries for maintenance functions. Therefore, the possibility of using cloud computing for running high performance computing applications is attractive. However, it has been shown elsewhere that current cloud computing platforms are not suitable for running some of these kinds of applications since the performance offered is very poor. The reason is mainly the overhead from virtualisation which is extensively used by most cloud computing platforms as a means to optimise resource usage. Furthermore, running HPC applications in current cloud platforms is a complex task that in many cases requires configuring a cluster of virtual machines (VMs). In this paper, we present a lightweight virtualisation approach for efficiently running the Weather Research and Forecasting (WRF) model (a computing- and communication-intensive application) in a cloud computing environment. Our approach also provides a higher-level programming model that automates the process of configuring a cluster of VMs. We assume such a cloud environment can be shared with other types of HPC applications such as mpiBLAST (an embarrassingly parallel application), and MiniFE (a memory-intensive application). Our experimental results show that lightweight virtualisation imposes about 5 % overhead and it substantially outperforms traditional heavyweight virtualisation such as KVM.
KW - Cloud computing
KW - High performance computing
KW - Lightweight virtual machines
KW - Virtualisation
KW - WRF model
UR - http://www.scopus.com/inward/record.url?scp=84961124571&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84961124571&partnerID=8YFLogxK
U2 - 10.1007/s12145-016-0253-7
DO - 10.1007/s12145-016-0253-7
M3 - Article
AN - SCOPUS:84961124571
SN - 1865-0473
VL - 9
SP - 365
EP - 382
JO - Earth Science Informatics
JF - Earth Science Informatics
IS - 3
ER -