Efficient execution of the WRF model and other HPC applications in the cloud

Hector A. Duran-Limon, Jesus Flores-Contreras, Nikos Parlavantzas, Ming Zhao, Angel Meulenert-Peña

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

There are many scientific applications that have high performance computing (HPC) demands. Such demands are traditionally supported by cluster- or Grid-based systems. Cloud computing, which has experienced a tremendous growth, emerged as an approach to provide on-demand access to computing resources. The cloud computing paradigm offers a number of advantages over other distributed platforms. For example, the access to resources is flexible and cost-effective since it is not necessary to invest a large amount of money on a computing infrastructure nor pay salaries for maintenance functions. Therefore, the possibility of using cloud computing for running high performance computing applications is attractive. However, it has been shown elsewhere that current cloud computing platforms are not suitable for running some of these kinds of applications since the performance offered is very poor. The reason is mainly the overhead from virtualisation which is extensively used by most cloud computing platforms as a means to optimise resource usage. Furthermore, running HPC applications in current cloud platforms is a complex task that in many cases requires configuring a cluster of virtual machines (VMs). In this paper, we present a lightweight virtualisation approach for efficiently running the Weather Research and Forecasting (WRF) model (a computing- and communication-intensive application) in a cloud computing environment. Our approach also provides a higher-level programming model that automates the process of configuring a cluster of VMs. We assume such a cloud environment can be shared with other types of HPC applications such as mpiBLAST (an embarrassingly parallel application), and MiniFE (a memory-intensive application). Our experimental results show that lightweight virtualisation imposes about 5 % overhead and it substantially outperforms traditional heavyweight virtualisation such as KVM.

Original languageEnglish (US)
Pages (from-to)365-382
Number of pages18
JournalEarth Science Informatics
Volume9
Issue number3
DOIs
StatePublished - Sep 1 2016
Externally publishedYes

Keywords

  • Cloud computing
  • High performance computing
  • Lightweight virtual machines
  • Virtualisation
  • WRF model

ASJC Scopus subject areas

  • General Earth and Planetary Sciences

Fingerprint

Dive into the research topics of 'Efficient execution of the WRF model and other HPC applications in the cloud'. Together they form a unique fingerprint.

Cite this