A hybrid parallel computing model to support scalable processing of big oceanographic spatial data

Miaomiao Song, WenWen Li, Wenqing Li, Enxiao Liu, Dingfeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Oceanographic sciences are facing big challenges due to the deluge of big data. As ofa 2010, the amount of new data stored in the world main countries, led by the US, has grown over 7 exabytes. Although the computer hardware is quickly evolving, with faster processor frequency, multi-core technology, and larger memory, traditional reprocessing paradigm on a single-desktop basis still suffers from significant limitations in its low computational efficiency and scalability. In this paper, we report our effort in developing a hybrid parallel computing model which utilizes Graphic Processing Unit (GPU) to accelerate Hadoop Map Reduce system. In each computing node, the actual reprocessing is offloaded from a CPU to a GPU to further boost up the system performance. We describe the architecture design of the proposed model and the automated task/data assignment on each GPU-enabled compute node. Electronic Navigational Charts in ocean fields involves a huge amount of spatio-temporal data. Reprojection of these data between different coordinate reference systems, which is a computation-intensive task, is selected as the use case. Systematic experiments were conducted to demonstrate the good performance of the proposed model.

Original languageEnglish (US)
Title of host publicationGeo-Spatial Knowledge and Intelligence - 4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016, Revised Selected Papers
PublisherSpringer Verlag
Pages276-285
Number of pages10
Volume699
ISBN (Print)9789811039683
DOIs
StatePublished - 2017
Event4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016 - Kowloon, Hong Kong
Duration: Nov 18 2016Nov 20 2016

Publication series

NameCommunications in Computer and Information Science
Volume699
ISSN (Print)18650929

Other

Other4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016
CountryHong Kong
City Kowloon
Period11/18/1611/20/16

Keywords

  • Coordinate projection
  • GPU general computing
  • Hadoop MapReduce
  • Oceanographic spatial data
  • Parallel computing

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'A hybrid parallel computing model to support scalable processing of big oceanographic spatial data'. Together they form a unique fingerprint.

  • Cite this

    Song, M., Li, W., Li, W., Liu, E., & Yu, D. (2017). A hybrid parallel computing model to support scalable processing of big oceanographic spatial data. In Geo-Spatial Knowledge and Intelligence - 4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016, Revised Selected Papers (Vol. 699, pp. 276-285). (Communications in Computer and Information Science; Vol. 699). Springer Verlag. https://doi.org/10.1007/978-981-10-3969-0_32