Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach

Yilin Wang, Yuheng Hu, Subbarao Kambhampati, Baoxin Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Scopus citations

Abstract

In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higherlevel challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th International Conference on Web and Social Media, ICWSM 2015
PublisherAAAI Press
Pages473-482
Number of pages10
ISBN (Print)9781577357339
StatePublished - 2015
Event9th International Conference on Web and Social Media, ICWSM 2015 - Oxford, United Kingdom
Duration: May 26 2015May 29 2015

Other

Other9th International Conference on Web and Social Media, ICWSM 2015
CountryUnited Kingdom
CityOxford
Period5/26/155/29/15

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach'. Together they form a unique fingerprint.

  • Cite this

    Wang, Y., Hu, Y., Kambhampati, S., & Li, B. (2015). Inferring sentiment from web images with joint inference on visual and social cues: A regulated matrix factorization approach. In Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015 (pp. 473-482). AAAI Press.