On the necessity of fine-tuned convolutional neural networks for medical imaging

Nima Tajbakhsh, Jae Y. Shin, Suryakanth R. Gurudu, R. Todd Hurst, Christopher B. Kendall, Michael B. Gotway, Jianming Liang

Research output: Chapter in Book/Report/Conference proceedingChapter

5 Scopus citations


This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.

Original languageEnglish (US)
Title of host publicationAdvances in Computer Vision and Pattern Recognition
PublisherSpringer London
Number of pages13
StatePublished - 2017

Publication series

NameAdvances in Computer Vision and Pattern Recognition
ISSN (Print)2191-6586
ISSN (Electronic)2191-6594


ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Todd Hurst, R., Kendall, C. B., Gotway, M. B., & Liang, J. (2017). On the necessity of fine-tuned convolutional neural networks for medical imaging. In Advances in Computer Vision and Pattern Recognition (9783319429984 ed., pp. 181-193). (Advances in Computer Vision and Pattern Recognition; No. 9783319429984). Springer London. https://doi.org/10.1007/978-3-319-42999-1_11