On the necessity of fine-tuned convolutional neural networks for medical imaging

Nima Tajbakhsh, Jae Y. Shin, Suryakanth R. Gurudu, R. Todd Hurst, Christopher B. Kendall, Michael B. Gotway, Jianming Liang

Research output: Chapter in Book/Report/Conference proceedingChapter

4 Citations (Scopus)

Abstract

This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.

Original languageEnglish (US)
Title of host publicationAdvances in Computer Vision and Pattern Recognition
PublisherSpringer London
Pages181-193
Number of pages13
Edition9783319429984
DOIs
StatePublished - 2017

Publication series

NameAdvances in Computer Vision and Pattern Recognition
Number9783319429984
ISSN (Print)2191-6586
ISSN (Electronic)2191-6594

Fingerprint

Medical imaging
Tuning
Neural networks

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Todd Hurst, R., Kendall, C. B., Gotway, M. B., & Liang, J. (2017). On the necessity of fine-tuned convolutional neural networks for medical imaging. In Advances in Computer Vision and Pattern Recognition (9783319429984 ed., pp. 181-193). (Advances in Computer Vision and Pattern Recognition; No. 9783319429984). Springer London. https://doi.org/10.1007/978-3-319-42999-1_11

On the necessity of fine-tuned convolutional neural networks for medical imaging. / Tajbakhsh, Nima; Shin, Jae Y.; Gurudu, Suryakanth R.; Todd Hurst, R.; Kendall, Christopher B.; Gotway, Michael B.; Liang, Jianming.

Advances in Computer Vision and Pattern Recognition. 9783319429984. ed. Springer London, 2017. p. 181-193 (Advances in Computer Vision and Pattern Recognition; No. 9783319429984).

Research output: Chapter in Book/Report/Conference proceedingChapter

Tajbakhsh, N, Shin, JY, Gurudu, SR, Todd Hurst, R, Kendall, CB, Gotway, MB & Liang, J 2017, On the necessity of fine-tuned convolutional neural networks for medical imaging. in Advances in Computer Vision and Pattern Recognition. 9783319429984 edn, Advances in Computer Vision and Pattern Recognition, no. 9783319429984, Springer London, pp. 181-193. https://doi.org/10.1007/978-3-319-42999-1_11
Tajbakhsh N, Shin JY, Gurudu SR, Todd Hurst R, Kendall CB, Gotway MB et al. On the necessity of fine-tuned convolutional neural networks for medical imaging. In Advances in Computer Vision and Pattern Recognition. 9783319429984 ed. Springer London. 2017. p. 181-193. (Advances in Computer Vision and Pattern Recognition; 9783319429984). https://doi.org/10.1007/978-3-319-42999-1_11
Tajbakhsh, Nima ; Shin, Jae Y. ; Gurudu, Suryakanth R. ; Todd Hurst, R. ; Kendall, Christopher B. ; Gotway, Michael B. ; Liang, Jianming. / On the necessity of fine-tuned convolutional neural networks for medical imaging. Advances in Computer Vision and Pattern Recognition. 9783319429984. ed. Springer London, 2017. pp. 181-193 (Advances in Computer Vision and Pattern Recognition; 9783319429984).
@inbook{2a0283fedefb40bf9640d1862d2f2aff,
title = "On the necessity of fine-tuned convolutional neural networks for medical imaging",
abstract = "This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.",
author = "Nima Tajbakhsh and Shin, {Jae Y.} and Gurudu, {Suryakanth R.} and {Todd Hurst}, R. and Kendall, {Christopher B.} and Gotway, {Michael B.} and Jianming Liang",
year = "2017",
doi = "10.1007/978-3-319-42999-1_11",
language = "English (US)",
series = "Advances in Computer Vision and Pattern Recognition",
publisher = "Springer London",
number = "9783319429984",
pages = "181--193",
booktitle = "Advances in Computer Vision and Pattern Recognition",
address = "United Kingdom",
edition = "9783319429984",

}

TY - CHAP

T1 - On the necessity of fine-tuned convolutional neural networks for medical imaging

AU - Tajbakhsh, Nima

AU - Shin, Jae Y.

AU - Gurudu, Suryakanth R.

AU - Todd Hurst, R.

AU - Kendall, Christopher B.

AU - Gotway, Michael B.

AU - Liang, Jianming

PY - 2017

Y1 - 2017

N2 - This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.

AB - This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.

UR - http://www.scopus.com/inward/record.url?scp=85024497663&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85024497663&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-42999-1_11

DO - 10.1007/978-3-319-42999-1_11

M3 - Chapter

T3 - Advances in Computer Vision and Pattern Recognition

SP - 181

EP - 193

BT - Advances in Computer Vision and Pattern Recognition

PB - Springer London

ER -