14 Citations (Scopus)

Abstract

In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th International Conference on Web and Social Media, ICWSM 2015
PublisherAAAI Press
Pages473-482
Number of pages10
ISBN (Print)9781577357339
StatePublished - 2015
Event9th International Conference on Web and Social Media, ICWSM 2015 - Oxford, United Kingdom
Duration: May 26 2015May 29 2015

Other

Other9th International Conference on Web and Social Media, ICWSM 2015
CountryUnited Kingdom
CityOxford
Period5/26/155/29/15

Fingerprint

Factorization
Labeling
Internet
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Wang, Y., Hu, Y., Kambhampati, S., & Li, B. (2015). Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach. In Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015 (pp. 473-482). AAAI Press.

Inferring sentiment from web images with joint inference on visual and social cues : A regulated matrix factorization approach. / Wang, Yilin; Hu, Yuheng; Kambhampati, Subbarao; Li, Baoxin.

Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015. AAAI Press, 2015. p. 473-482.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Y, Hu, Y, Kambhampati, S & Li, B 2015, Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach. in Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015. AAAI Press, pp. 473-482, 9th International Conference on Web and Social Media, ICWSM 2015, Oxford, United Kingdom, 5/26/15.
Wang Y, Hu Y, Kambhampati S, Li B. Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach. In Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015. AAAI Press. 2015. p. 473-482
Wang, Yilin ; Hu, Yuheng ; Kambhampati, Subbarao ; Li, Baoxin. / Inferring sentiment from web images with joint inference on visual and social cues : A regulated matrix factorization approach. Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015. AAAI Press, 2015. pp. 473-482
@inproceedings{cbfc0e5d9e6f4ad482f890c039648f01,
title = "Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach",
abstract = "In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.",
author = "Yilin Wang and Yuheng Hu and Subbarao Kambhampati and Baoxin Li",
year = "2015",
language = "English (US)",
isbn = "9781577357339",
pages = "473--482",
booktitle = "Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015",
publisher = "AAAI Press",

}

TY - GEN

T1 - Inferring sentiment from web images with joint inference on visual and social cues

T2 - A regulated matrix factorization approach

AU - Wang, Yilin

AU - Hu, Yuheng

AU - Kambhampati, Subbarao

AU - Li, Baoxin

PY - 2015

Y1 - 2015

N2 - In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.

AB - In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.

UR - http://www.scopus.com/inward/record.url?scp=84960959840&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960959840&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781577357339

SP - 473

EP - 482

BT - Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015

PB - AAAI Press

ER -