Retrieving and organizing web pages by "information unit"

Since WWW encourages hypertext and hypermedia docu-ment authoring (e.g., HTML or XML), Web authors tend to create documents that are composed of multiple pages con-nected with hyperlinks or frames. A Web document may be authored in multiple ways, such as (1) all information in one physical page, or (2) a main page and the related information in separate linked pages. Existing Web search engines, however, return only physical pages. In this paper, we introduce and describe the use of the concept of informa-tion unit, which can be viewed as a logical Web document consisting of multiple physical pages as one atomic retrieval unit. We present an algorithm to eÆciently retrieve infor-mation units. Our algorithm can perform progressive query processing over a Web index by considering both document semantic similarity and link structures. Experimental re-sults on synthetic graphs and real Web data show the ef-fectiveness and usefulness of the proposed information unit retrieval technique.