Graph based multi-modality learning

Hanghang Tong, Jingrui He, Mingjing Li, Changshui Zhang, Wei Ying Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

82 Scopus citations

Abstract

To better understand the content of multimedia, a lot of research efforts have been made on how to learn from multi-modal feature. In this paper, it is studied from a graph point of view: each kind of feature from one modality is represented as one independent graph; and the learning task is formulated as inferring from the constraints in every graph as well as supervision information (if available). For semi-supervised learning, two different fusion schemes, namely linear form and sequential form, are proposed. For each scheme, it is derived from optimization point of view; and further justified from two sides: similarity propagation and Bayesian interpretation. By doing so, we reveal the regular optimization nature, transductive learning nature as well as prior fusion nature of the proposed schemes, respectively. Moreover, the proposed method can be easily extended to unsupervised learning, including clustering and embedding. Systematic experimental results validate the effectiveness of the proposed method.

Original languageEnglish (US)
Title of host publicationProceedings of the 13th ACM International Conference on Multimedia, MM 2005
Pages862-871
Number of pages10
DOIs
StatePublished - Dec 1 2005
Event13th ACM International Conference on Multimedia, MM 2005 - Singapore, Singapore
Duration: Nov 6 2005Nov 11 2005

Publication series

NameProceedings of the 13th ACM International Conference on Multimedia, MM 2005

Conference

Conference13th ACM International Conference on Multimedia, MM 2005
CountrySingapore
CitySingapore
Period11/6/0511/11/05

    Fingerprint

Keywords

  • Bayesian interpretation
  • Graph model
  • Multi-modality analysis
  • Regularized optimization
  • Similarity propagation

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Cite this

Tong, H., He, J., Li, M., Zhang, C., & Ma, W. Y. (2005). Graph based multi-modality learning. In Proceedings of the 13th ACM International Conference on Multimedia, MM 2005 (pp. 862-871). (Proceedings of the 13th ACM International Conference on Multimedia, MM 2005). https://doi.org/10.1145/1101149.1101337