TY - GEN
T1 - A hybrid parallel computing model to support scalable processing of big oceanographic spatial data
AU - Song, Miaomiao
AU - Li, WenWen
AU - Li, Wenqing
AU - Liu, Enxiao
AU - Yu, Dingfeng
N1 - Funding Information:
The research work report in this paper was mainly supported by the Young Scientists Funds (Grant No. 2015QN027) from Shandong Academy of Sciences. It was partially sponsored by the Youth Fund of Natural Science of China (Grant No. 41401435).
Publisher Copyright:
© Springer Nature Singapore Pte Ltd. 2017.
PY - 2017
Y1 - 2017
N2 - Oceanographic sciences are facing big challenges due to the deluge of big data. As ofa 2010, the amount of new data stored in the world main countries, led by the US, has grown over 7 exabytes. Although the computer hardware is quickly evolving, with faster processor frequency, multi-core technology, and larger memory, traditional reprocessing paradigm on a single-desktop basis still suffers from significant limitations in its low computational efficiency and scalability. In this paper, we report our effort in developing a hybrid parallel computing model which utilizes Graphic Processing Unit (GPU) to accelerate Hadoop Map Reduce system. In each computing node, the actual reprocessing is offloaded from a CPU to a GPU to further boost up the system performance. We describe the architecture design of the proposed model and the automated task/data assignment on each GPU-enabled compute node. Electronic Navigational Charts in ocean fields involves a huge amount of spatio-temporal data. Reprojection of these data between different coordinate reference systems, which is a computation-intensive task, is selected as the use case. Systematic experiments were conducted to demonstrate the good performance of the proposed model.
AB - Oceanographic sciences are facing big challenges due to the deluge of big data. As ofa 2010, the amount of new data stored in the world main countries, led by the US, has grown over 7 exabytes. Although the computer hardware is quickly evolving, with faster processor frequency, multi-core technology, and larger memory, traditional reprocessing paradigm on a single-desktop basis still suffers from significant limitations in its low computational efficiency and scalability. In this paper, we report our effort in developing a hybrid parallel computing model which utilizes Graphic Processing Unit (GPU) to accelerate Hadoop Map Reduce system. In each computing node, the actual reprocessing is offloaded from a CPU to a GPU to further boost up the system performance. We describe the architecture design of the proposed model and the automated task/data assignment on each GPU-enabled compute node. Electronic Navigational Charts in ocean fields involves a huge amount of spatio-temporal data. Reprojection of these data between different coordinate reference systems, which is a computation-intensive task, is selected as the use case. Systematic experiments were conducted to demonstrate the good performance of the proposed model.
KW - Coordinate projection
KW - GPU general computing
KW - Hadoop MapReduce
KW - Oceanographic spatial data
KW - Parallel computing
UR - http://www.scopus.com/inward/record.url?scp=85014879160&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85014879160&partnerID=8YFLogxK
U2 - 10.1007/978-981-10-3969-0_32
DO - 10.1007/978-981-10-3969-0_32
M3 - Conference contribution
AN - SCOPUS:85014879160
SN - 9789811039683
T3 - Communications in Computer and Information Science
SP - 276
EP - 285
BT - Geo-Spatial Knowledge and Intelligence - 4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016, Revised Selected Papers
A2 - Yuan, Hanning
A2 - Geng, Jing
A2 - Bian, Fuling
PB - Springer Verlag
T2 - 4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016
Y2 - 18 November 2016 through 20 November 2016
ER -