Deep multimodality model for multi-task multi-view learning

Lecheng Zheng, Yu Cheng, Jingrui He

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Many real-world problems exhibit the coexistence of multiple types of heterogeneity, such as view heterogeneity (i.e., multi-view property) and task heterogeneity (i.e., multi-task property). For example, in an image classification problem containing multiple poses of the same object, each pose can be considered as one view, and the detection of each type of object can be treated as one task. Furthermore, in some problems, the data type of multiple views might be different. In a web classification problem, for instance, we might be provided an image and text mixed data set, where the web pages are characterized by both images and texts. A common strategy to solve this kind of problem is to leverage the consistency of views and the relatedness of tasks to build the prediction model. In the context of deep neural network, multitask relatedness is usually realized by grouping tasks at each layer, while multi-view consistency is usually enforced by finding the maximal correlation coefficient between views. However, there is no existing deep learning algorithm that jointly models task and view dual heterogeneity, particularly for a data set with multiple modalities (text and image mixed data set or text and video mixed data set, etc.). In this paper, we bridge this gap by proposing a deep multi-task multi-view learning framework that learns a deep representation for such dual-heterogeneity problems. Empirical studies on multiple real-world data sets demonstrate the effectiveness of our proposed Deep-MTMV algorithm.

Original languageEnglish (US)
Title of host publicationSIAM International Conference on Data Mining, SDM 2019
PublisherSociety for Industrial and Applied Mathematics Publications
Pages10-16
Number of pages7
ISBN (Electronic)9781611975673
StatePublished - Jan 1 2019
Event19th SIAM International Conference on Data Mining, SDM 2019 - Calgary, Canada
Duration: May 2 2019May 4 2019

Publication series

NameSIAM International Conference on Data Mining, SDM 2019

Conference

Conference19th SIAM International Conference on Data Mining, SDM 2019
CountryCanada
CityCalgary
Period5/2/195/4/19

Fingerprint

Image classification
Learning algorithms
Websites
Deep neural networks
Deep learning

Keywords

  • Deep learning
  • Multi-task learning
  • Multi-view learning

ASJC Scopus subject areas

  • Software

Cite this

Zheng, L., Cheng, Y., & He, J. (2019). Deep multimodality model for multi-task multi-view learning. In SIAM International Conference on Data Mining, SDM 2019 (pp. 10-16). (SIAM International Conference on Data Mining, SDM 2019). Society for Industrial and Applied Mathematics Publications.

Deep multimodality model for multi-task multi-view learning. / Zheng, Lecheng; Cheng, Yu; He, Jingrui.

SIAM International Conference on Data Mining, SDM 2019. Society for Industrial and Applied Mathematics Publications, 2019. p. 10-16 (SIAM International Conference on Data Mining, SDM 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zheng, L, Cheng, Y & He, J 2019, Deep multimodality model for multi-task multi-view learning. in SIAM International Conference on Data Mining, SDM 2019. SIAM International Conference on Data Mining, SDM 2019, Society for Industrial and Applied Mathematics Publications, pp. 10-16, 19th SIAM International Conference on Data Mining, SDM 2019, Calgary, Canada, 5/2/19.
Zheng L, Cheng Y, He J. Deep multimodality model for multi-task multi-view learning. In SIAM International Conference on Data Mining, SDM 2019. Society for Industrial and Applied Mathematics Publications. 2019. p. 10-16. (SIAM International Conference on Data Mining, SDM 2019).
Zheng, Lecheng ; Cheng, Yu ; He, Jingrui. / Deep multimodality model for multi-task multi-view learning. SIAM International Conference on Data Mining, SDM 2019. Society for Industrial and Applied Mathematics Publications, 2019. pp. 10-16 (SIAM International Conference on Data Mining, SDM 2019).
@inproceedings{ca0988793d0a4caa9978b128eb1b4404,
title = "Deep multimodality model for multi-task multi-view learning",
abstract = "Many real-world problems exhibit the coexistence of multiple types of heterogeneity, such as view heterogeneity (i.e., multi-view property) and task heterogeneity (i.e., multi-task property). For example, in an image classification problem containing multiple poses of the same object, each pose can be considered as one view, and the detection of each type of object can be treated as one task. Furthermore, in some problems, the data type of multiple views might be different. In a web classification problem, for instance, we might be provided an image and text mixed data set, where the web pages are characterized by both images and texts. A common strategy to solve this kind of problem is to leverage the consistency of views and the relatedness of tasks to build the prediction model. In the context of deep neural network, multitask relatedness is usually realized by grouping tasks at each layer, while multi-view consistency is usually enforced by finding the maximal correlation coefficient between views. However, there is no existing deep learning algorithm that jointly models task and view dual heterogeneity, particularly for a data set with multiple modalities (text and image mixed data set or text and video mixed data set, etc.). In this paper, we bridge this gap by proposing a deep multi-task multi-view learning framework that learns a deep representation for such dual-heterogeneity problems. Empirical studies on multiple real-world data sets demonstrate the effectiveness of our proposed Deep-MTMV algorithm.",
keywords = "Deep learning, Multi-task learning, Multi-view learning",
author = "Lecheng Zheng and Yu Cheng and Jingrui He",
year = "2019",
month = "1",
day = "1",
language = "English (US)",
series = "SIAM International Conference on Data Mining, SDM 2019",
publisher = "Society for Industrial and Applied Mathematics Publications",
pages = "10--16",
booktitle = "SIAM International Conference on Data Mining, SDM 2019",

}

TY - GEN

T1 - Deep multimodality model for multi-task multi-view learning

AU - Zheng, Lecheng

AU - Cheng, Yu

AU - He, Jingrui

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Many real-world problems exhibit the coexistence of multiple types of heterogeneity, such as view heterogeneity (i.e., multi-view property) and task heterogeneity (i.e., multi-task property). For example, in an image classification problem containing multiple poses of the same object, each pose can be considered as one view, and the detection of each type of object can be treated as one task. Furthermore, in some problems, the data type of multiple views might be different. In a web classification problem, for instance, we might be provided an image and text mixed data set, where the web pages are characterized by both images and texts. A common strategy to solve this kind of problem is to leverage the consistency of views and the relatedness of tasks to build the prediction model. In the context of deep neural network, multitask relatedness is usually realized by grouping tasks at each layer, while multi-view consistency is usually enforced by finding the maximal correlation coefficient between views. However, there is no existing deep learning algorithm that jointly models task and view dual heterogeneity, particularly for a data set with multiple modalities (text and image mixed data set or text and video mixed data set, etc.). In this paper, we bridge this gap by proposing a deep multi-task multi-view learning framework that learns a deep representation for such dual-heterogeneity problems. Empirical studies on multiple real-world data sets demonstrate the effectiveness of our proposed Deep-MTMV algorithm.

AB - Many real-world problems exhibit the coexistence of multiple types of heterogeneity, such as view heterogeneity (i.e., multi-view property) and task heterogeneity (i.e., multi-task property). For example, in an image classification problem containing multiple poses of the same object, each pose can be considered as one view, and the detection of each type of object can be treated as one task. Furthermore, in some problems, the data type of multiple views might be different. In a web classification problem, for instance, we might be provided an image and text mixed data set, where the web pages are characterized by both images and texts. A common strategy to solve this kind of problem is to leverage the consistency of views and the relatedness of tasks to build the prediction model. In the context of deep neural network, multitask relatedness is usually realized by grouping tasks at each layer, while multi-view consistency is usually enforced by finding the maximal correlation coefficient between views. However, there is no existing deep learning algorithm that jointly models task and view dual heterogeneity, particularly for a data set with multiple modalities (text and image mixed data set or text and video mixed data set, etc.). In this paper, we bridge this gap by proposing a deep multi-task multi-view learning framework that learns a deep representation for such dual-heterogeneity problems. Empirical studies on multiple real-world data sets demonstrate the effectiveness of our proposed Deep-MTMV algorithm.

KW - Deep learning

KW - Multi-task learning

KW - Multi-view learning

UR - http://www.scopus.com/inward/record.url?scp=85066092321&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066092321&partnerID=8YFLogxK

M3 - Conference contribution

T3 - SIAM International Conference on Data Mining, SDM 2019

SP - 10

EP - 16

BT - SIAM International Conference on Data Mining, SDM 2019

PB - Society for Industrial and Applied Mathematics Publications

ER -