Skip-and-prune: Cosine-based top-k query processing for efficient context-sensitive document retrieval

Jong Wook Kim, Kasim Candan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

Keyword search and ranked retrieval together emerged as popular data access paradigms for various kinds of data, from web pages to XML and relational databases. A user can submit keywords without knowing much (sometimes nothing) about the complex structure underlying a data collection, yet the system can identify, rank, and return a set of relevant matches by exploiting statistics about the distribution and structure of the data. Keyword-based data models are also suitable for capturing user's search context in terms of weights associated to the keywords in the query. Given a search context, the data in the database can also be re-interpreted for semantically correct retrieval. This option, however, is often ignored as the cost of re-assessing the content in the database naively tends to be prohibitive. In this paper, we first argue that top-k query processing can help tackle this challenge by re-assessing only the relevant parts of the database, efficiently. A road-block in this process, however, is that most efficient implementations of top-k query processing assume that the scoring function is monotonic, whereas the cosine-based scoring function needed for re-interpretation of content based on user context is not. In this paper, we develop an efficient top-k query processing algorithm, skip-and-prune (SnP), which is able to process top-k queries under cosine-based non-monotonic scoring functions. We compare the use of proposed algorithm against the alternative implementations of the context-aware retrieval, including naive top-k, accumulator-based inverted files, and full-scan. The experiment results show that while being fast, naive top-k is not an effective solution due to the non-monotonicity of underlying scoring function. The proposed technique, SnP, however, matches the precision of accumulator-based inverted files and full-scan, yet it is orders of magnitude faster than these.

Original languageEnglish (US)
Title of host publicationSIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems
Pages115-126
Number of pages12
DOIs
StatePublished - 2009
EventInternational Conference on Management of Data and 28th Symposium on Principles of Database Systems, SIGMOD-PODS'09 - Providence, RI, United States
Duration: Jun 29 2009Jul 2 2009

Other

OtherInternational Conference on Management of Data and 28th Symposium on Principles of Database Systems, SIGMOD-PODS'09
CountryUnited States
CityProvidence, RI
Period6/29/097/2/09

Fingerprint

Query processing
XML
Data structures
Websites
Statistics
Costs
Experiments

Keywords

  • Ranking
  • Top-K

ASJC Scopus subject areas

  • Software

Cite this

Kim, J. W., & Candan, K. (2009). Skip-and-prune: Cosine-based top-k query processing for efficient context-sensitive document retrieval. In SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems (pp. 115-126) https://doi.org/10.1145/1559845.1559859

Skip-and-prune : Cosine-based top-k query processing for efficient context-sensitive document retrieval. / Kim, Jong Wook; Candan, Kasim.

SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems. 2009. p. 115-126.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, JW & Candan, K 2009, Skip-and-prune: Cosine-based top-k query processing for efficient context-sensitive document retrieval. in SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems. pp. 115-126, International Conference on Management of Data and 28th Symposium on Principles of Database Systems, SIGMOD-PODS'09, Providence, RI, United States, 6/29/09. https://doi.org/10.1145/1559845.1559859
Kim JW, Candan K. Skip-and-prune: Cosine-based top-k query processing for efficient context-sensitive document retrieval. In SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems. 2009. p. 115-126 https://doi.org/10.1145/1559845.1559859
Kim, Jong Wook ; Candan, Kasim. / Skip-and-prune : Cosine-based top-k query processing for efficient context-sensitive document retrieval. SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems. 2009. pp. 115-126
@inproceedings{590e97c84d6046a9a1e3c64e525f5c8f,
title = "Skip-and-prune: Cosine-based top-k query processing for efficient context-sensitive document retrieval",
abstract = "Keyword search and ranked retrieval together emerged as popular data access paradigms for various kinds of data, from web pages to XML and relational databases. A user can submit keywords without knowing much (sometimes nothing) about the complex structure underlying a data collection, yet the system can identify, rank, and return a set of relevant matches by exploiting statistics about the distribution and structure of the data. Keyword-based data models are also suitable for capturing user's search context in terms of weights associated to the keywords in the query. Given a search context, the data in the database can also be re-interpreted for semantically correct retrieval. This option, however, is often ignored as the cost of re-assessing the content in the database naively tends to be prohibitive. In this paper, we first argue that top-k query processing can help tackle this challenge by re-assessing only the relevant parts of the database, efficiently. A road-block in this process, however, is that most efficient implementations of top-k query processing assume that the scoring function is monotonic, whereas the cosine-based scoring function needed for re-interpretation of content based on user context is not. In this paper, we develop an efficient top-k query processing algorithm, skip-and-prune (SnP), which is able to process top-k queries under cosine-based non-monotonic scoring functions. We compare the use of proposed algorithm against the alternative implementations of the context-aware retrieval, including naive top-k, accumulator-based inverted files, and full-scan. The experiment results show that while being fast, naive top-k is not an effective solution due to the non-monotonicity of underlying scoring function. The proposed technique, SnP, however, matches the precision of accumulator-based inverted files and full-scan, yet it is orders of magnitude faster than these.",
keywords = "Ranking, Top-K",
author = "Kim, {Jong Wook} and Kasim Candan",
year = "2009",
doi = "10.1145/1559845.1559859",
language = "English (US)",
isbn = "9781605585543",
pages = "115--126",
booktitle = "SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems",

}

TY - GEN

T1 - Skip-and-prune

T2 - Cosine-based top-k query processing for efficient context-sensitive document retrieval

AU - Kim, Jong Wook

AU - Candan, Kasim

PY - 2009

Y1 - 2009

N2 - Keyword search and ranked retrieval together emerged as popular data access paradigms for various kinds of data, from web pages to XML and relational databases. A user can submit keywords without knowing much (sometimes nothing) about the complex structure underlying a data collection, yet the system can identify, rank, and return a set of relevant matches by exploiting statistics about the distribution and structure of the data. Keyword-based data models are also suitable for capturing user's search context in terms of weights associated to the keywords in the query. Given a search context, the data in the database can also be re-interpreted for semantically correct retrieval. This option, however, is often ignored as the cost of re-assessing the content in the database naively tends to be prohibitive. In this paper, we first argue that top-k query processing can help tackle this challenge by re-assessing only the relevant parts of the database, efficiently. A road-block in this process, however, is that most efficient implementations of top-k query processing assume that the scoring function is monotonic, whereas the cosine-based scoring function needed for re-interpretation of content based on user context is not. In this paper, we develop an efficient top-k query processing algorithm, skip-and-prune (SnP), which is able to process top-k queries under cosine-based non-monotonic scoring functions. We compare the use of proposed algorithm against the alternative implementations of the context-aware retrieval, including naive top-k, accumulator-based inverted files, and full-scan. The experiment results show that while being fast, naive top-k is not an effective solution due to the non-monotonicity of underlying scoring function. The proposed technique, SnP, however, matches the precision of accumulator-based inverted files and full-scan, yet it is orders of magnitude faster than these.

AB - Keyword search and ranked retrieval together emerged as popular data access paradigms for various kinds of data, from web pages to XML and relational databases. A user can submit keywords without knowing much (sometimes nothing) about the complex structure underlying a data collection, yet the system can identify, rank, and return a set of relevant matches by exploiting statistics about the distribution and structure of the data. Keyword-based data models are also suitable for capturing user's search context in terms of weights associated to the keywords in the query. Given a search context, the data in the database can also be re-interpreted for semantically correct retrieval. This option, however, is often ignored as the cost of re-assessing the content in the database naively tends to be prohibitive. In this paper, we first argue that top-k query processing can help tackle this challenge by re-assessing only the relevant parts of the database, efficiently. A road-block in this process, however, is that most efficient implementations of top-k query processing assume that the scoring function is monotonic, whereas the cosine-based scoring function needed for re-interpretation of content based on user context is not. In this paper, we develop an efficient top-k query processing algorithm, skip-and-prune (SnP), which is able to process top-k queries under cosine-based non-monotonic scoring functions. We compare the use of proposed algorithm against the alternative implementations of the context-aware retrieval, including naive top-k, accumulator-based inverted files, and full-scan. The experiment results show that while being fast, naive top-k is not an effective solution due to the non-monotonicity of underlying scoring function. The proposed technique, SnP, however, matches the precision of accumulator-based inverted files and full-scan, yet it is orders of magnitude faster than these.

KW - Ranking

KW - Top-K

UR - http://www.scopus.com/inward/record.url?scp=70849092408&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70849092408&partnerID=8YFLogxK

U2 - 10.1145/1559845.1559859

DO - 10.1145/1559845.1559859

M3 - Conference contribution

AN - SCOPUS:70849092408

SN - 9781605585543

SP - 115

EP - 126

BT - SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems

ER -