A feature transfer enabled multi-task deep learning model on medical imaging

Object detection, segmentation, and classification are three common tasks in medical image analysis. Multi-task deep learning (MTL) tackles these three tasks jointly, which provides two advantages—saving computational cost and improving robustness against overfitting. Existing multi-task deep models start with learning each task as an individual objective in parallel and then integrate the tasks at the end of the architecture with one cost function. Such architecture fails to take advantage of the combined power of the features from each individual task at an early stage of the training. In this research, we propose a new architecture, FT-MTL-Net, an MTL model enabled by feature transfer. Traditional transfer learning deals with the same or similar task (e.g., classification) from different data sources (a.k.a. domain). The underlying assumption is that the knowledge gained from various source domains may help the learning task on the target domain. Our proposed FT-MTL-Net utilizes the different tasks from the same domain. Considering that features from the tasks are different views of the domain, the combined feature maps can be well exploited using knowledge from multiple views to enhance the generalizability. To evaluate the validity of the proposed approach, FT-MTL-Net is compared with models from literature including eight classification models, four detection models, and three segmentation models using a publicly available Full Filed Digital Mammogram dataset for breast cancer diagnosis. Experimental results show that the proposed FT-MTL-Net outperforms the competing models in classification and detection and has comparable results in segmentation.