Collaborative scientific workflow composition as a service- An infrastructure supporting collaborative data analytics workflow design and management

Jia Zhang, Qihao Bao, Xiaoyi Duan, Shiyong Lu, Lijun Xue, Runyu Shi, Pingbo Tang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

The need for collaborative data analytics increases significantly when confronted with the challenges of big data. Although workflow tools offer a formal way to define, automate, and repeat multi-step computational procedures, designing complex data processing workflow requires collaboration from multiple people with complementary expertise. Existing tools are not suitable to support collaborative design of comprehensive workflows. To address such a challenge, this paper reports the design and development of a software infrastructure with the capability of supporting collaborative data-oriented workflow composition and management, adding a key component to existing cyberinfrastructure that will support big data collaboration through the Internet. A collaborative provenance query model (CPM) is presented together with graph-based patterns and algebra. A hypergraph theory-based provenance mining technique is reported. The research extends an existing opensource workflow tool, by adding system-level facilities to support human interaction and cooperation that are essential for an effective and efficient scientific collaboration.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE 2nd International Conference on Collaboration and Internet Computing, IEEE CIC 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages219-228
Number of pages10
ISBN (Electronic)9781509046072
DOIs
StatePublished - Jan 6 2017
Event2nd IEEE International Conference on Collaboration and Internet Computing, IEEE CIC 2016 - Pittsburgh, United States
Duration: Nov 1 2016Nov 3 2016

Publication series

NameProceedings - 2016 IEEE 2nd International Conference on Collaboration and Internet Computing, IEEE CIC 2016

Other

Other2nd IEEE International Conference on Collaboration and Internet Computing, IEEE CIC 2016
Country/TerritoryUnited States
CityPittsburgh
Period11/1/1611/3/16

Keywords

  • Big data analytics
  • Collaborative provenance
  • Collaborative workflow design
  • Scientific workflow

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Safety, Risk, Reliability and Quality
  • Sociology and Political Science

Fingerprint

Dive into the research topics of 'Collaborative scientific workflow composition as a service- An infrastructure supporting collaborative data analytics workflow design and management'. Together they form a unique fingerprint.

Cite this