Abstract

Pointwise label and pairwise label are both widely used in computer vision tasks. For example, supervised image classification and annotation approaches use pointwise label, while attribute-based image relative learning often adopts pairwise labels. These two types of labels are often considered independently and most existing efforts utilize them separately. However, pointwise labels in image classification and tag annotation are inherently related to the pairwise labels. For example, an image labeled with "coast" and annotated with "beach, sea, sand, sky" is more likely to have a higher ranking score in terms of the attribute "open"; while "men shoes" ranked highly on the attribute "formal" are likely to be annotated with "leather, lace up" than "buckle, fabric". The existence of potential relations between pointwise labels and pairwise labels motivates us to fuse them together for jointly addressing related vision tasks. In particular, we provide a principled way to capture the relations between class labels, tags and attributes; and propose a novel framework PPP(Pointwise and Pairwise image label Prediction), which is based on overlapped group structure extracted from the pointwise-pairwise-label bipartite graph. With experiments on benchmark datasets, we demonstrate that the proposed framework achieves superior performance on three vision tasks compared to the state-of-the-art methods.

Original languageEnglish (US)
Title of host publication2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
PublisherIEEE Computer Society
Pages6005-6013
Number of pages9
Volume2016-January
ISBN (Electronic)9781467388511
StatePublished - 2016
Event2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 - Las Vegas, United States
Duration: Jun 26 2016Jul 1 2016

Other

Other2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
CountryUnited States
CityLas Vegas
Period6/26/167/1/16

Fingerprint

Labels
Image classification
Leather
Electric fuses
Beaches
Computer vision
Coastal zones
Sand

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Wang, Y., Wang, S., Tang, J., Liu, H., & Li, B. (2016). PPP: Joint pointwise and pairwise image label prediction. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 (Vol. 2016-January, pp. 6005-6013). IEEE Computer Society.

PPP : Joint pointwise and pairwise image label prediction. / Wang, Yilin; Wang, Suhang; Tang, Jiliang; Liu, Huan; Li, Baoxin.

2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. Vol. 2016-January IEEE Computer Society, 2016. p. 6005-6013.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Y, Wang, S, Tang, J, Liu, H & Li, B 2016, PPP: Joint pointwise and pairwise image label prediction. in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. vol. 2016-January, IEEE Computer Society, pp. 6005-6013, 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, United States, 6/26/16.
Wang Y, Wang S, Tang J, Liu H, Li B. PPP: Joint pointwise and pairwise image label prediction. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. Vol. 2016-January. IEEE Computer Society. 2016. p. 6005-6013
Wang, Yilin ; Wang, Suhang ; Tang, Jiliang ; Liu, Huan ; Li, Baoxin. / PPP : Joint pointwise and pairwise image label prediction. 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. Vol. 2016-January IEEE Computer Society, 2016. pp. 6005-6013
@inproceedings{695301535f4148048f66ff59919983a5,
title = "PPP: Joint pointwise and pairwise image label prediction",
abstract = "Pointwise label and pairwise label are both widely used in computer vision tasks. For example, supervised image classification and annotation approaches use pointwise label, while attribute-based image relative learning often adopts pairwise labels. These two types of labels are often considered independently and most existing efforts utilize them separately. However, pointwise labels in image classification and tag annotation are inherently related to the pairwise labels. For example, an image labeled with {"}coast{"} and annotated with {"}beach, sea, sand, sky{"} is more likely to have a higher ranking score in terms of the attribute {"}open{"}; while {"}men shoes{"} ranked highly on the attribute {"}formal{"} are likely to be annotated with {"}leather, lace up{"} than {"}buckle, fabric{"}. The existence of potential relations between pointwise labels and pairwise labels motivates us to fuse them together for jointly addressing related vision tasks. In particular, we provide a principled way to capture the relations between class labels, tags and attributes; and propose a novel framework PPP(Pointwise and Pairwise image label Prediction), which is based on overlapped group structure extracted from the pointwise-pairwise-label bipartite graph. With experiments on benchmark datasets, we demonstrate that the proposed framework achieves superior performance on three vision tasks compared to the state-of-the-art methods.",
author = "Yilin Wang and Suhang Wang and Jiliang Tang and Huan Liu and Baoxin Li",
year = "2016",
language = "English (US)",
volume = "2016-January",
pages = "6005--6013",
booktitle = "2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016",
publisher = "IEEE Computer Society",
address = "United States",

}

TY - GEN

T1 - PPP

T2 - Joint pointwise and pairwise image label prediction

AU - Wang, Yilin

AU - Wang, Suhang

AU - Tang, Jiliang

AU - Liu, Huan

AU - Li, Baoxin

PY - 2016

Y1 - 2016

N2 - Pointwise label and pairwise label are both widely used in computer vision tasks. For example, supervised image classification and annotation approaches use pointwise label, while attribute-based image relative learning often adopts pairwise labels. These two types of labels are often considered independently and most existing efforts utilize them separately. However, pointwise labels in image classification and tag annotation are inherently related to the pairwise labels. For example, an image labeled with "coast" and annotated with "beach, sea, sand, sky" is more likely to have a higher ranking score in terms of the attribute "open"; while "men shoes" ranked highly on the attribute "formal" are likely to be annotated with "leather, lace up" than "buckle, fabric". The existence of potential relations between pointwise labels and pairwise labels motivates us to fuse them together for jointly addressing related vision tasks. In particular, we provide a principled way to capture the relations between class labels, tags and attributes; and propose a novel framework PPP(Pointwise and Pairwise image label Prediction), which is based on overlapped group structure extracted from the pointwise-pairwise-label bipartite graph. With experiments on benchmark datasets, we demonstrate that the proposed framework achieves superior performance on three vision tasks compared to the state-of-the-art methods.

AB - Pointwise label and pairwise label are both widely used in computer vision tasks. For example, supervised image classification and annotation approaches use pointwise label, while attribute-based image relative learning often adopts pairwise labels. These two types of labels are often considered independently and most existing efforts utilize them separately. However, pointwise labels in image classification and tag annotation are inherently related to the pairwise labels. For example, an image labeled with "coast" and annotated with "beach, sea, sand, sky" is more likely to have a higher ranking score in terms of the attribute "open"; while "men shoes" ranked highly on the attribute "formal" are likely to be annotated with "leather, lace up" than "buckle, fabric". The existence of potential relations between pointwise labels and pairwise labels motivates us to fuse them together for jointly addressing related vision tasks. In particular, we provide a principled way to capture the relations between class labels, tags and attributes; and propose a novel framework PPP(Pointwise and Pairwise image label Prediction), which is based on overlapped group structure extracted from the pointwise-pairwise-label bipartite graph. With experiments on benchmark datasets, we demonstrate that the proposed framework achieves superior performance on three vision tasks compared to the state-of-the-art methods.

UR - http://www.scopus.com/inward/record.url?scp=84986321986&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84986321986&partnerID=8YFLogxK

M3 - Conference contribution

VL - 2016-January

SP - 6005

EP - 6013

BT - 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016

PB - IEEE Computer Society

ER -