Deep model based transfer and multi-task learning for biological image analysis

Wenlu Zhang, Rongjian Li, Tao Zeng, Qian Sun, Sudhir Kumar, Jieping Ye, Shuiwang Ji

Research output: Chapter in Book/Report/Conference proceedingConference contribution

34 Citations (Scopus)

Abstract

A central theme in learning from image data is to develop appropriate image representations for the specific task at hand. Traditional methods used handcrafted local features combined with high-level image representations to generate image-level representations. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila melanogaster, texture features based on wavelets were particularly effective for determining the developmental stages from in situ hybridization (ISH) images. Such image representation is however not suitable for controlled vocabulary (CV) term annotation because each CV term is often associated with only a part of an image. Here, we developed problem-independent feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks (CNNs) that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain and used directly as feature extractors to compute image representations. Furthermore, we employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images, and also extracted features from the fine-tuned models. Experimental results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges. We also demonstrated that the intermediate layers of deep models produced the best gene expression pattern representations.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
PublisherAssociation for Computing Machinery
Pages1475-1484
Number of pages10
Volume2015-August
ISBN (Print)9781450336642
DOIs
StatePublished - Aug 10 2015
Externally publishedYes
Event21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2015 - Sydney, Australia
Duration: Aug 10 2015Aug 13 2015

Other

Other21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2015
CountryAustralia
CitySydney
Period8/10/158/13/15

Fingerprint

Image analysis
Gene expression
Thesauri
Feature extraction
Textures
Pixels
Neural networks

Keywords

  • Bioinformatics
  • Deep learning
  • Image analysis
  • Multi-task learning
  • Transfer learning

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Zhang, W., Li, R., Zeng, T., Sun, Q., Kumar, S., Ye, J., & Ji, S. (2015). Deep model based transfer and multi-task learning for biological image analysis. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Vol. 2015-August, pp. 1475-1484). Association for Computing Machinery. https://doi.org/10.1145/2783258.2783304

Deep model based transfer and multi-task learning for biological image analysis. / Zhang, Wenlu; Li, Rongjian; Zeng, Tao; Sun, Qian; Kumar, Sudhir; Ye, Jieping; Ji, Shuiwang.

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Vol. 2015-August Association for Computing Machinery, 2015. p. 1475-1484.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhang, W, Li, R, Zeng, T, Sun, Q, Kumar, S, Ye, J & Ji, S 2015, Deep model based transfer and multi-task learning for biological image analysis. in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. vol. 2015-August, Association for Computing Machinery, pp. 1475-1484, 21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2015, Sydney, Australia, 8/10/15. https://doi.org/10.1145/2783258.2783304
Zhang W, Li R, Zeng T, Sun Q, Kumar S, Ye J et al. Deep model based transfer and multi-task learning for biological image analysis. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Vol. 2015-August. Association for Computing Machinery. 2015. p. 1475-1484 https://doi.org/10.1145/2783258.2783304
Zhang, Wenlu ; Li, Rongjian ; Zeng, Tao ; Sun, Qian ; Kumar, Sudhir ; Ye, Jieping ; Ji, Shuiwang. / Deep model based transfer and multi-task learning for biological image analysis. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Vol. 2015-August Association for Computing Machinery, 2015. pp. 1475-1484
@inproceedings{3a31834dc8d64b6e8aaa581e707b4c67,
title = "Deep model based transfer and multi-task learning for biological image analysis",
abstract = "A central theme in learning from image data is to develop appropriate image representations for the specific task at hand. Traditional methods used handcrafted local features combined with high-level image representations to generate image-level representations. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila melanogaster, texture features based on wavelets were particularly effective for determining the developmental stages from in situ hybridization (ISH) images. Such image representation is however not suitable for controlled vocabulary (CV) term annotation because each CV term is often associated with only a part of an image. Here, we developed problem-independent feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks (CNNs) that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain and used directly as feature extractors to compute image representations. Furthermore, we employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images, and also extracted features from the fine-tuned models. Experimental results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges. We also demonstrated that the intermediate layers of deep models produced the best gene expression pattern representations.",
keywords = "Bioinformatics, Deep learning, Image analysis, Multi-task learning, Transfer learning",
author = "Wenlu Zhang and Rongjian Li and Tao Zeng and Qian Sun and Sudhir Kumar and Jieping Ye and Shuiwang Ji",
year = "2015",
month = "8",
day = "10",
doi = "10.1145/2783258.2783304",
language = "English (US)",
isbn = "9781450336642",
volume = "2015-August",
pages = "1475--1484",
booktitle = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Deep model based transfer and multi-task learning for biological image analysis

AU - Zhang, Wenlu

AU - Li, Rongjian

AU - Zeng, Tao

AU - Sun, Qian

AU - Kumar, Sudhir

AU - Ye, Jieping

AU - Ji, Shuiwang

PY - 2015/8/10

Y1 - 2015/8/10

N2 - A central theme in learning from image data is to develop appropriate image representations for the specific task at hand. Traditional methods used handcrafted local features combined with high-level image representations to generate image-level representations. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila melanogaster, texture features based on wavelets were particularly effective for determining the developmental stages from in situ hybridization (ISH) images. Such image representation is however not suitable for controlled vocabulary (CV) term annotation because each CV term is often associated with only a part of an image. Here, we developed problem-independent feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks (CNNs) that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain and used directly as feature extractors to compute image representations. Furthermore, we employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images, and also extracted features from the fine-tuned models. Experimental results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges. We also demonstrated that the intermediate layers of deep models produced the best gene expression pattern representations.

AB - A central theme in learning from image data is to develop appropriate image representations for the specific task at hand. Traditional methods used handcrafted local features combined with high-level image representations to generate image-level representations. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila melanogaster, texture features based on wavelets were particularly effective for determining the developmental stages from in situ hybridization (ISH) images. Such image representation is however not suitable for controlled vocabulary (CV) term annotation because each CV term is often associated with only a part of an image. Here, we developed problem-independent feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks (CNNs) that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain and used directly as feature extractors to compute image representations. Furthermore, we employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images, and also extracted features from the fine-tuned models. Experimental results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges. We also demonstrated that the intermediate layers of deep models produced the best gene expression pattern representations.

KW - Bioinformatics

KW - Deep learning

KW - Image analysis

KW - Multi-task learning

KW - Transfer learning

UR - http://www.scopus.com/inward/record.url?scp=84954161664&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84954161664&partnerID=8YFLogxK

U2 - 10.1145/2783258.2783304

DO - 10.1145/2783258.2783304

M3 - Conference contribution

AN - SCOPUS:84954161664

SN - 9781450336642

VL - 2015-August

SP - 1475

EP - 1484

BT - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

PB - Association for Computing Machinery

ER -