TY - GEN
T1 - Inferring sentiment from web images with joint inference on visual and social cues
T2 - 9th International Conference on Web and Social Media, ICWSM 2015
AU - Wang, Yilin
AU - Hu, Yuheng
AU - Kambhampati, Subbarao
AU - Li, Baoxin
N1 - Publisher Copyright:
© Copyright 2015, Association for the Advancement of Artificial Intelligence. All rights reserved.
PY - 2015
Y1 - 2015
N2 - In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.
AB - In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.
UR - http://www.scopus.com/inward/record.url?scp=84960959840&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84960959840&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84960959840
T3 - Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015
SP - 473
EP - 482
BT - Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015
PB - AAAI press
Y2 - 26 May 2015 through 29 May 2015
ER -