Query processing over incomplete autonomous databases: Query rewriting using learned data dependencies

Garrett Wolf, Aravind Kalavagattu, Hemal Khatri, Raju Balakrishnan, Bhaumik Chokshi, Jianchun Fan, Yi Chen, Subbarao Kambhampati

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Incompleteness due to missing attribute values (aka "null values") is very common in autonomous web databases, on which user accesses are usually supported through mediators. Traditional query processing techniques that focus on the strict soundness of answer tuples often ignore tuples with critical missing attributes, even if they wind up being relevant to a user query. Ideally we would like the mediator to retrieve such possibleanswers and gauge their relevance by accessing their likelihood of being pertinent answers to the query. The autonomous nature of web databases poses several challenges in realizing this objective. Such challenges include the restricted access privileges imposed on the data, the limited support for query patterns, and the bounded pool of database and network resources in the web environment. We introduce a novel query rewriting and optimization framework QPIAD that tackles these challenges. Our technique involves reformulating the user query based on mined correlations among the database attributes. The reformulated queries are aimed at retrieving the relevant possibleanswers in addition to the certain answers. QPIAD is able to gauge the relevance of such queries allowing tradeoffs in reducing the costs of database query processing and answer transmission. To support this framework, we develop methods for mining attribute correlations (in terms of Approximate Functional Dependencies), value distributions (in the form of Naïve Bayes Classifiers), and selectivity estimates. We present empirical studies to demonstrate that our approach is able to effectively retrieve relevant possibleanswers with high precision, high recall, and manageable cost.

Original languageEnglish (US)
Pages (from-to)1167-1190
Number of pages24
JournalVLDB Journal
Volume18
Issue number5
DOIs
StatePublished - Oct 2009

Fingerprint

Query processing
Gages
Costs
Classifiers

Keywords

  • Incomplete databases
  • Query rewriting
  • Uncertainty

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems

Cite this

Query processing over incomplete autonomous databases : Query rewriting using learned data dependencies. / Wolf, Garrett; Kalavagattu, Aravind; Khatri, Hemal; Balakrishnan, Raju; Chokshi, Bhaumik; Fan, Jianchun; Chen, Yi; Kambhampati, Subbarao.

In: VLDB Journal, Vol. 18, No. 5, 10.2009, p. 1167-1190.

Research output: Contribution to journalArticle

Wolf, G, Kalavagattu, A, Khatri, H, Balakrishnan, R, Chokshi, B, Fan, J, Chen, Y & Kambhampati, S 2009, 'Query processing over incomplete autonomous databases: Query rewriting using learned data dependencies', VLDB Journal, vol. 18, no. 5, pp. 1167-1190. https://doi.org/10.1007/s00778-009-0155-0
Wolf, Garrett ; Kalavagattu, Aravind ; Khatri, Hemal ; Balakrishnan, Raju ; Chokshi, Bhaumik ; Fan, Jianchun ; Chen, Yi ; Kambhampati, Subbarao. / Query processing over incomplete autonomous databases : Query rewriting using learned data dependencies. In: VLDB Journal. 2009 ; Vol. 18, No. 5. pp. 1167-1190.
@article{37c6a8439e1b43f08fd27ee86e41c869,
title = "Query processing over incomplete autonomous databases: Query rewriting using learned data dependencies",
abstract = "Incompleteness due to missing attribute values (aka {"}null values{"}) is very common in autonomous web databases, on which user accesses are usually supported through mediators. Traditional query processing techniques that focus on the strict soundness of answer tuples often ignore tuples with critical missing attributes, even if they wind up being relevant to a user query. Ideally we would like the mediator to retrieve such possibleanswers and gauge their relevance by accessing their likelihood of being pertinent answers to the query. The autonomous nature of web databases poses several challenges in realizing this objective. Such challenges include the restricted access privileges imposed on the data, the limited support for query patterns, and the bounded pool of database and network resources in the web environment. We introduce a novel query rewriting and optimization framework QPIAD that tackles these challenges. Our technique involves reformulating the user query based on mined correlations among the database attributes. The reformulated queries are aimed at retrieving the relevant possibleanswers in addition to the certain answers. QPIAD is able to gauge the relevance of such queries allowing tradeoffs in reducing the costs of database query processing and answer transmission. To support this framework, we develop methods for mining attribute correlations (in terms of Approximate Functional Dependencies), value distributions (in the form of Na{\"i}ve Bayes Classifiers), and selectivity estimates. We present empirical studies to demonstrate that our approach is able to effectively retrieve relevant possibleanswers with high precision, high recall, and manageable cost.",
keywords = "Incomplete databases, Query rewriting, Uncertainty",
author = "Garrett Wolf and Aravind Kalavagattu and Hemal Khatri and Raju Balakrishnan and Bhaumik Chokshi and Jianchun Fan and Yi Chen and Subbarao Kambhampati",
year = "2009",
month = "10",
doi = "10.1007/s00778-009-0155-0",
language = "English (US)",
volume = "18",
pages = "1167--1190",
journal = "VLDB Journal",
issn = "1066-8888",
publisher = "Springer New York",
number = "5",

}

TY - JOUR

T1 - Query processing over incomplete autonomous databases

T2 - Query rewriting using learned data dependencies

AU - Wolf, Garrett

AU - Kalavagattu, Aravind

AU - Khatri, Hemal

AU - Balakrishnan, Raju

AU - Chokshi, Bhaumik

AU - Fan, Jianchun

AU - Chen, Yi

AU - Kambhampati, Subbarao

PY - 2009/10

Y1 - 2009/10

N2 - Incompleteness due to missing attribute values (aka "null values") is very common in autonomous web databases, on which user accesses are usually supported through mediators. Traditional query processing techniques that focus on the strict soundness of answer tuples often ignore tuples with critical missing attributes, even if they wind up being relevant to a user query. Ideally we would like the mediator to retrieve such possibleanswers and gauge their relevance by accessing their likelihood of being pertinent answers to the query. The autonomous nature of web databases poses several challenges in realizing this objective. Such challenges include the restricted access privileges imposed on the data, the limited support for query patterns, and the bounded pool of database and network resources in the web environment. We introduce a novel query rewriting and optimization framework QPIAD that tackles these challenges. Our technique involves reformulating the user query based on mined correlations among the database attributes. The reformulated queries are aimed at retrieving the relevant possibleanswers in addition to the certain answers. QPIAD is able to gauge the relevance of such queries allowing tradeoffs in reducing the costs of database query processing and answer transmission. To support this framework, we develop methods for mining attribute correlations (in terms of Approximate Functional Dependencies), value distributions (in the form of Naïve Bayes Classifiers), and selectivity estimates. We present empirical studies to demonstrate that our approach is able to effectively retrieve relevant possibleanswers with high precision, high recall, and manageable cost.

AB - Incompleteness due to missing attribute values (aka "null values") is very common in autonomous web databases, on which user accesses are usually supported through mediators. Traditional query processing techniques that focus on the strict soundness of answer tuples often ignore tuples with critical missing attributes, even if they wind up being relevant to a user query. Ideally we would like the mediator to retrieve such possibleanswers and gauge their relevance by accessing their likelihood of being pertinent answers to the query. The autonomous nature of web databases poses several challenges in realizing this objective. Such challenges include the restricted access privileges imposed on the data, the limited support for query patterns, and the bounded pool of database and network resources in the web environment. We introduce a novel query rewriting and optimization framework QPIAD that tackles these challenges. Our technique involves reformulating the user query based on mined correlations among the database attributes. The reformulated queries are aimed at retrieving the relevant possibleanswers in addition to the certain answers. QPIAD is able to gauge the relevance of such queries allowing tradeoffs in reducing the costs of database query processing and answer transmission. To support this framework, we develop methods for mining attribute correlations (in terms of Approximate Functional Dependencies), value distributions (in the form of Naïve Bayes Classifiers), and selectivity estimates. We present empirical studies to demonstrate that our approach is able to effectively retrieve relevant possibleanswers with high precision, high recall, and manageable cost.

KW - Incomplete databases

KW - Query rewriting

KW - Uncertainty

UR - http://www.scopus.com/inward/record.url?scp=70349835611&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70349835611&partnerID=8YFLogxK

U2 - 10.1007/s00778-009-0155-0

DO - 10.1007/s00778-009-0155-0

M3 - Article

AN - SCOPUS:70349835611

VL - 18

SP - 1167

EP - 1190

JO - VLDB Journal

JF - VLDB Journal

SN - 1066-8888

IS - 5

ER -