Abstract

The problem of finding interesting tuples in a data set, more commonly known as the skyline problem, has been extensively studied in scenarios where the data is static. More recently, skyline research has moved towards data streaming environments, where tuples arrive/expire in a continuous manner. Several algorithms have been developed to track skyline changes over sliding windows; however, existing methods focus on skyline analysis in which all required skyline attributes belong to a single incoming data stream. This constraint renders current algorithms unsuitable for applications that require a real-time "join" operation to be carried out between multiple incoming data streams, arriving from different sources, before the skyline query can be answered. Based on this motivation, in this paper, we address the problem of computing skyline-window-join (SWJ) queries over pairs of data streams, considering sliding windows that take into account only the most recent tuples. In particular, we propose a Layered Skyline-window-Join (LSJ) operator that (a) partitions the overall process into processing layers and (b) maintains skyline-join results in an incremental manner by continuously monitoring the changes in all layers of the process. We combine the advantages of existing skyline methods (including those that efficiently maintain skyline results over a single stream, and those that compute the skyline of pairs of static data sets) to develop a novel iteration-fabric skyline-window-join processing structure. Using the iteration-fabric, LSJ eliminates redundant work across consecutive windows by leveraging shared data across all iteration layers of the windowed skyline-join processing. To the best of our knowledge, this is the first paper that addresses join-based skyline queries over sliding windows. Extensive experimental evaluations over real and simulated data show that LSJ provides large gains over naive extensions of existing schemes which are not designed to eliminate redundant work across multiple processing layers.

Original languageEnglish (US)
Title of host publicationProceedings - International Conference on Data Engineering
Pages985-996
Number of pages12
DOIs
StatePublished - 2013
Event29th International Conference on Data Engineering, ICDE 2013 - Brisbane, QLD, Australia
Duration: Apr 8 2013Apr 11 2013

Other

Other29th International Conference on Data Engineering, ICDE 2013
CountryAustralia
CityBrisbane, QLD
Period4/8/134/11/13

Fingerprint

Processing
Monitoring

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing
  • Software

Cite this

Nagendra, M., & Candan, K. (2013). Layered processing of skyline-window-join (SWJ) queries using iteration-fabric. In Proceedings - International Conference on Data Engineering (pp. 985-996). [6544891] https://doi.org/10.1109/ICDE.2013.6544891

Layered processing of skyline-window-join (SWJ) queries using iteration-fabric. / Nagendra, Mithila; Candan, Kasim.

Proceedings - International Conference on Data Engineering. 2013. p. 985-996 6544891.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nagendra, M & Candan, K 2013, Layered processing of skyline-window-join (SWJ) queries using iteration-fabric. in Proceedings - International Conference on Data Engineering., 6544891, pp. 985-996, 29th International Conference on Data Engineering, ICDE 2013, Brisbane, QLD, Australia, 4/8/13. https://doi.org/10.1109/ICDE.2013.6544891
Nagendra M, Candan K. Layered processing of skyline-window-join (SWJ) queries using iteration-fabric. In Proceedings - International Conference on Data Engineering. 2013. p. 985-996. 6544891 https://doi.org/10.1109/ICDE.2013.6544891
Nagendra, Mithila ; Candan, Kasim. / Layered processing of skyline-window-join (SWJ) queries using iteration-fabric. Proceedings - International Conference on Data Engineering. 2013. pp. 985-996
@inproceedings{6d50bffeb5a541698ea568e8da8ec877,
title = "Layered processing of skyline-window-join (SWJ) queries using iteration-fabric",
abstract = "The problem of finding interesting tuples in a data set, more commonly known as the skyline problem, has been extensively studied in scenarios where the data is static. More recently, skyline research has moved towards data streaming environments, where tuples arrive/expire in a continuous manner. Several algorithms have been developed to track skyline changes over sliding windows; however, existing methods focus on skyline analysis in which all required skyline attributes belong to a single incoming data stream. This constraint renders current algorithms unsuitable for applications that require a real-time {"}join{"} operation to be carried out between multiple incoming data streams, arriving from different sources, before the skyline query can be answered. Based on this motivation, in this paper, we address the problem of computing skyline-window-join (SWJ) queries over pairs of data streams, considering sliding windows that take into account only the most recent tuples. In particular, we propose a Layered Skyline-window-Join (LSJ) operator that (a) partitions the overall process into processing layers and (b) maintains skyline-join results in an incremental manner by continuously monitoring the changes in all layers of the process. We combine the advantages of existing skyline methods (including those that efficiently maintain skyline results over a single stream, and those that compute the skyline of pairs of static data sets) to develop a novel iteration-fabric skyline-window-join processing structure. Using the iteration-fabric, LSJ eliminates redundant work across consecutive windows by leveraging shared data across all iteration layers of the windowed skyline-join processing. To the best of our knowledge, this is the first paper that addresses join-based skyline queries over sliding windows. Extensive experimental evaluations over real and simulated data show that LSJ provides large gains over naive extensions of existing schemes which are not designed to eliminate redundant work across multiple processing layers.",
author = "Mithila Nagendra and Kasim Candan",
year = "2013",
doi = "10.1109/ICDE.2013.6544891",
language = "English (US)",
isbn = "9781467349086",
pages = "985--996",
booktitle = "Proceedings - International Conference on Data Engineering",

}

TY - GEN

T1 - Layered processing of skyline-window-join (SWJ) queries using iteration-fabric

AU - Nagendra, Mithila

AU - Candan, Kasim

PY - 2013

Y1 - 2013

N2 - The problem of finding interesting tuples in a data set, more commonly known as the skyline problem, has been extensively studied in scenarios where the data is static. More recently, skyline research has moved towards data streaming environments, where tuples arrive/expire in a continuous manner. Several algorithms have been developed to track skyline changes over sliding windows; however, existing methods focus on skyline analysis in which all required skyline attributes belong to a single incoming data stream. This constraint renders current algorithms unsuitable for applications that require a real-time "join" operation to be carried out between multiple incoming data streams, arriving from different sources, before the skyline query can be answered. Based on this motivation, in this paper, we address the problem of computing skyline-window-join (SWJ) queries over pairs of data streams, considering sliding windows that take into account only the most recent tuples. In particular, we propose a Layered Skyline-window-Join (LSJ) operator that (a) partitions the overall process into processing layers and (b) maintains skyline-join results in an incremental manner by continuously monitoring the changes in all layers of the process. We combine the advantages of existing skyline methods (including those that efficiently maintain skyline results over a single stream, and those that compute the skyline of pairs of static data sets) to develop a novel iteration-fabric skyline-window-join processing structure. Using the iteration-fabric, LSJ eliminates redundant work across consecutive windows by leveraging shared data across all iteration layers of the windowed skyline-join processing. To the best of our knowledge, this is the first paper that addresses join-based skyline queries over sliding windows. Extensive experimental evaluations over real and simulated data show that LSJ provides large gains over naive extensions of existing schemes which are not designed to eliminate redundant work across multiple processing layers.

AB - The problem of finding interesting tuples in a data set, more commonly known as the skyline problem, has been extensively studied in scenarios where the data is static. More recently, skyline research has moved towards data streaming environments, where tuples arrive/expire in a continuous manner. Several algorithms have been developed to track skyline changes over sliding windows; however, existing methods focus on skyline analysis in which all required skyline attributes belong to a single incoming data stream. This constraint renders current algorithms unsuitable for applications that require a real-time "join" operation to be carried out between multiple incoming data streams, arriving from different sources, before the skyline query can be answered. Based on this motivation, in this paper, we address the problem of computing skyline-window-join (SWJ) queries over pairs of data streams, considering sliding windows that take into account only the most recent tuples. In particular, we propose a Layered Skyline-window-Join (LSJ) operator that (a) partitions the overall process into processing layers and (b) maintains skyline-join results in an incremental manner by continuously monitoring the changes in all layers of the process. We combine the advantages of existing skyline methods (including those that efficiently maintain skyline results over a single stream, and those that compute the skyline of pairs of static data sets) to develop a novel iteration-fabric skyline-window-join processing structure. Using the iteration-fabric, LSJ eliminates redundant work across consecutive windows by leveraging shared data across all iteration layers of the windowed skyline-join processing. To the best of our knowledge, this is the first paper that addresses join-based skyline queries over sliding windows. Extensive experimental evaluations over real and simulated data show that LSJ provides large gains over naive extensions of existing schemes which are not designed to eliminate redundant work across multiple processing layers.

UR - http://www.scopus.com/inward/record.url?scp=84881336326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84881336326&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2013.6544891

DO - 10.1109/ICDE.2013.6544891

M3 - Conference contribution

SN - 9781467349086

SP - 985

EP - 996

BT - Proceedings - International Conference on Data Engineering

ER -