Enabling composite applications through an asynchronous shared memory interface

Douglas Otstott, Noah Evans, Latchesar Ionkov, Ming Zhao, Michael Lang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

In this work we address the growing need for mechanisms for intranode application composition. We provide a novel shared memory interface that allows composite applications, two or more coupled applications, to share internal data structures without blocking. This allows independent progress of the applications such that they can proceed in a parallel, overlapped fashion. Composite applications using in-node shared memory can reduce the amount of data to be communicated between nodes, allowing data reduction or analytics to be performed locally and in parallel. To validate our approach we implemented our solution in Linux and used two proxy-applications to demonstrate how applications can be coupled and compare the performance to a traditional solution. We also compared the impact of composite applications to the performance of their unmodified versions. Our solution incurs small overhead in HPC Linux environments and significantly outperforms preexisting approaches.

Original languageEnglish (US)
Title of host publicationProceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages219-224
Number of pages6
ISBN (Print)9781479956654
DOIs
StatePublished - Jan 7 2015
Externally publishedYes
Event2nd IEEE International Conference on Big Data, IEEE Big Data 2014 - Washington, United States
Duration: Oct 27 2014Oct 30 2014

Other

Other2nd IEEE International Conference on Big Data, IEEE Big Data 2014
CountryUnited States
CityWashington
Period10/27/1410/30/14

Fingerprint

Interfaces (computer)
Data storage equipment
Composite materials
Data structures
Data reduction
Chemical analysis

Keywords

  • checkpoint
  • composite applications
  • memory management
  • operating systems
  • shared memory

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems

Cite this

Otstott, D., Evans, N., Ionkov, L., Zhao, M., & Lang, M. (2015). Enabling composite applications through an asynchronous shared memory interface. In Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014 (pp. 219-224). [7004236] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2014.7004236

Enabling composite applications through an asynchronous shared memory interface. / Otstott, Douglas; Evans, Noah; Ionkov, Latchesar; Zhao, Ming; Lang, Michael.

Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. Institute of Electrical and Electronics Engineers Inc., 2015. p. 219-224 7004236.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Otstott, D, Evans, N, Ionkov, L, Zhao, M & Lang, M 2015, Enabling composite applications through an asynchronous shared memory interface. in Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014., 7004236, Institute of Electrical and Electronics Engineers Inc., pp. 219-224, 2nd IEEE International Conference on Big Data, IEEE Big Data 2014, Washington, United States, 10/27/14. https://doi.org/10.1109/BigData.2014.7004236
Otstott D, Evans N, Ionkov L, Zhao M, Lang M. Enabling composite applications through an asynchronous shared memory interface. In Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. Institute of Electrical and Electronics Engineers Inc. 2015. p. 219-224. 7004236 https://doi.org/10.1109/BigData.2014.7004236
Otstott, Douglas ; Evans, Noah ; Ionkov, Latchesar ; Zhao, Ming ; Lang, Michael. / Enabling composite applications through an asynchronous shared memory interface. Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 219-224
@inproceedings{ea55600b4c214868b8f3ac8e5874a10a,
title = "Enabling composite applications through an asynchronous shared memory interface",
abstract = "In this work we address the growing need for mechanisms for intranode application composition. We provide a novel shared memory interface that allows composite applications, two or more coupled applications, to share internal data structures without blocking. This allows independent progress of the applications such that they can proceed in a parallel, overlapped fashion. Composite applications using in-node shared memory can reduce the amount of data to be communicated between nodes, allowing data reduction or analytics to be performed locally and in parallel. To validate our approach we implemented our solution in Linux and used two proxy-applications to demonstrate how applications can be coupled and compare the performance to a traditional solution. We also compared the impact of composite applications to the performance of their unmodified versions. Our solution incurs small overhead in HPC Linux environments and significantly outperforms preexisting approaches.",
keywords = "checkpoint, composite applications, memory management, operating systems, shared memory",
author = "Douglas Otstott and Noah Evans and Latchesar Ionkov and Ming Zhao and Michael Lang",
year = "2015",
month = "1",
day = "7",
doi = "10.1109/BigData.2014.7004236",
language = "English (US)",
isbn = "9781479956654",
pages = "219--224",
booktitle = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Enabling composite applications through an asynchronous shared memory interface

AU - Otstott, Douglas

AU - Evans, Noah

AU - Ionkov, Latchesar

AU - Zhao, Ming

AU - Lang, Michael

PY - 2015/1/7

Y1 - 2015/1/7

N2 - In this work we address the growing need for mechanisms for intranode application composition. We provide a novel shared memory interface that allows composite applications, two or more coupled applications, to share internal data structures without blocking. This allows independent progress of the applications such that they can proceed in a parallel, overlapped fashion. Composite applications using in-node shared memory can reduce the amount of data to be communicated between nodes, allowing data reduction or analytics to be performed locally and in parallel. To validate our approach we implemented our solution in Linux and used two proxy-applications to demonstrate how applications can be coupled and compare the performance to a traditional solution. We also compared the impact of composite applications to the performance of their unmodified versions. Our solution incurs small overhead in HPC Linux environments and significantly outperforms preexisting approaches.

AB - In this work we address the growing need for mechanisms for intranode application composition. We provide a novel shared memory interface that allows composite applications, two or more coupled applications, to share internal data structures without blocking. This allows independent progress of the applications such that they can proceed in a parallel, overlapped fashion. Composite applications using in-node shared memory can reduce the amount of data to be communicated between nodes, allowing data reduction or analytics to be performed locally and in parallel. To validate our approach we implemented our solution in Linux and used two proxy-applications to demonstrate how applications can be coupled and compare the performance to a traditional solution. We also compared the impact of composite applications to the performance of their unmodified versions. Our solution incurs small overhead in HPC Linux environments and significantly outperforms preexisting approaches.

KW - checkpoint

KW - composite applications

KW - memory management

KW - operating systems

KW - shared memory

UR - http://www.scopus.com/inward/record.url?scp=84921711670&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84921711670&partnerID=8YFLogxK

U2 - 10.1109/BigData.2014.7004236

DO - 10.1109/BigData.2014.7004236

M3 - Conference contribution

AN - SCOPUS:84921711670

SN - 9781479956654

SP - 219

EP - 224

BT - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

PB - Institute of Electrical and Electronics Engineers Inc.

ER -