Abstract

Data- and model-driven computer simulations are increasingly critical in many application domains. Yet, several critical data challenges remain in obtaining and leveraging simulations in decision making. Simulations may track 100s of parameters, spanning multiple layers and spatialtemporal frames, affected by complex inter-dependent dynamic processes. Moreover, due to the large numbers of unknowns, decision makers usually need to generate ensembles of stochastic realizations, requiring 10s-1000s of individual simulation instances. The situation on the ground evolves unpredictably, requiring continuously adaptive simulation ensembles. We introduce the DataStorm framework for simulation ensemble management, and demonstrate its DataStorm-FE data- and decisionow and coordination engine for creating and maintaining coupled, multi-model simulation ensembles. DataStorm-FE enables end-to-end ensemble planning and optimization, including parameter-space sampling, output aggregation and alignment, and state and provenance data management, to improve the overall simulation process. It also aims to work effciently, producing results while working within a limited simulation budget, and incorporates a multivariate, spatiotemporal data browser to empower decision-making based on these improved results.

Original languageEnglish (US)
Pages (from-to)1906-1909
Number of pages4
JournalProceedings of the VLDB Endowment
Volume11
Issue number12
DOIs
StatePublished - Jan 1 2017
Event44th International Conference on Very Large Data Bases, VLDB 2018 - Rio de Janeiro, Brazil
Duration: Aug 27 2017Aug 31 2017

Fingerprint

Decision making
Engines
Information management
Agglomeration
Sampling
Planning
Computer simulation

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

DataStormFE : A Data-and decision-flow and coordination engine for coupled simulation ensembles. / Behrens, Hans Walter; Candan, Kasim; Chen, Xilun; Gadkari, Ashish; Garg, Yash; Li, Mao Lin.

In: Proceedings of the VLDB Endowment, Vol. 11, No. 12, 01.01.2017, p. 1906-1909.

Research output: Contribution to journalConference article

Behrens, Hans Walter ; Candan, Kasim ; Chen, Xilun ; Gadkari, Ashish ; Garg, Yash ; Li, Mao Lin. / DataStormFE : A Data-and decision-flow and coordination engine for coupled simulation ensembles. In: Proceedings of the VLDB Endowment. 2017 ; Vol. 11, No. 12. pp. 1906-1909.
@article{4005ddd9e28444c5ac57823a7a95a85d,
title = "DataStormFE: A Data-and decision-flow and coordination engine for coupled simulation ensembles",
abstract = "Data- and model-driven computer simulations are increasingly critical in many application domains. Yet, several critical data challenges remain in obtaining and leveraging simulations in decision making. Simulations may track 100s of parameters, spanning multiple layers and spatialtemporal frames, affected by complex inter-dependent dynamic processes. Moreover, due to the large numbers of unknowns, decision makers usually need to generate ensembles of stochastic realizations, requiring 10s-1000s of individual simulation instances. The situation on the ground evolves unpredictably, requiring continuously adaptive simulation ensembles. We introduce the DataStorm framework for simulation ensemble management, and demonstrate its DataStorm-FE data- and decisionow and coordination engine for creating and maintaining coupled, multi-model simulation ensembles. DataStorm-FE enables end-to-end ensemble planning and optimization, including parameter-space sampling, output aggregation and alignment, and state and provenance data management, to improve the overall simulation process. It also aims to work effciently, producing results while working within a limited simulation budget, and incorporates a multivariate, spatiotemporal data browser to empower decision-making based on these improved results.",
author = "Behrens, {Hans Walter} and Kasim Candan and Xilun Chen and Ashish Gadkari and Yash Garg and Li, {Mao Lin}",
year = "2017",
month = "1",
day = "1",
doi = "10.14778/3229863.3236221",
language = "English (US)",
volume = "11",
pages = "1906--1909",
journal = "Proceedings of the VLDB Endowment",
issn = "2150-8097",
publisher = "Very Large Data Base Endowment Inc.",
number = "12",

}

TY - JOUR

T1 - DataStormFE

T2 - A Data-and decision-flow and coordination engine for coupled simulation ensembles

AU - Behrens, Hans Walter

AU - Candan, Kasim

AU - Chen, Xilun

AU - Gadkari, Ashish

AU - Garg, Yash

AU - Li, Mao Lin

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Data- and model-driven computer simulations are increasingly critical in many application domains. Yet, several critical data challenges remain in obtaining and leveraging simulations in decision making. Simulations may track 100s of parameters, spanning multiple layers and spatialtemporal frames, affected by complex inter-dependent dynamic processes. Moreover, due to the large numbers of unknowns, decision makers usually need to generate ensembles of stochastic realizations, requiring 10s-1000s of individual simulation instances. The situation on the ground evolves unpredictably, requiring continuously adaptive simulation ensembles. We introduce the DataStorm framework for simulation ensemble management, and demonstrate its DataStorm-FE data- and decisionow and coordination engine for creating and maintaining coupled, multi-model simulation ensembles. DataStorm-FE enables end-to-end ensemble planning and optimization, including parameter-space sampling, output aggregation and alignment, and state and provenance data management, to improve the overall simulation process. It also aims to work effciently, producing results while working within a limited simulation budget, and incorporates a multivariate, spatiotemporal data browser to empower decision-making based on these improved results.

AB - Data- and model-driven computer simulations are increasingly critical in many application domains. Yet, several critical data challenges remain in obtaining and leveraging simulations in decision making. Simulations may track 100s of parameters, spanning multiple layers and spatialtemporal frames, affected by complex inter-dependent dynamic processes. Moreover, due to the large numbers of unknowns, decision makers usually need to generate ensembles of stochastic realizations, requiring 10s-1000s of individual simulation instances. The situation on the ground evolves unpredictably, requiring continuously adaptive simulation ensembles. We introduce the DataStorm framework for simulation ensemble management, and demonstrate its DataStorm-FE data- and decisionow and coordination engine for creating and maintaining coupled, multi-model simulation ensembles. DataStorm-FE enables end-to-end ensemble planning and optimization, including parameter-space sampling, output aggregation and alignment, and state and provenance data management, to improve the overall simulation process. It also aims to work effciently, producing results while working within a limited simulation budget, and incorporates a multivariate, spatiotemporal data browser to empower decision-making based on these improved results.

UR - http://www.scopus.com/inward/record.url?scp=85058882679&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058882679&partnerID=8YFLogxK

U2 - 10.14778/3229863.3236221

DO - 10.14778/3229863.3236221

M3 - Conference article

VL - 11

SP - 1906

EP - 1909

JO - Proceedings of the VLDB Endowment

JF - Proceedings of the VLDB Endowment

SN - 2150-8097

IS - 12

ER -