Convolutional neural networks: Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images

Mohammadhassan Izadyyazdanabadi, Evgenii Belykh, Michael Mooney, Nikolay Martirosyan, Jennifer Eschbacher, Peter Nakaji, Mark C. Preul, Yezhou Yang

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence technology undergoing assessment for applications in brain tumor surgery. Many of the CLE images can be distorted and interpreted as nondiagnostic. However, just one neat CLE image might suffice for intraoperative diagnosis of the tumor. While manual examination of thousands of nondiagnostic images during surgery would be impractical, this creates an opportunity for a model to select diagnostic images for the pathologists or surgeons review. In this study, we sought to develop a deep learning model to automatically detect the diagnostic images. We explored the effect of training regimes and ensemble modeling and localized histological features from diagnostic CLE images. The developed model could achieve promising agreement with the ground truth. With the speed and precision of the proposed method, it has potential to be integrated into the operative workflow in the brain tumor surgery.

Original languageEnglish (US)
Pages (from-to)10-20
Number of pages11
JournalJournal of Visual Communication and Image Representation
Volume54
DOIs
StatePublished - Jul 1 2018

Fingerprint

Tuning
Semantics
Surgery
Neural networks
Tumors
Lasers
Brain
Fluorescence

Keywords

  • Brain tumor
  • Confocal laser endomicroscopy
  • Ensemble modeling
  • Neural network
  • Surgical vision
  • Unsupervised localization

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Cite this

Convolutional neural networks : Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images. / Izadyyazdanabadi, Mohammadhassan; Belykh, Evgenii; Mooney, Michael; Martirosyan, Nikolay; Eschbacher, Jennifer; Nakaji, Peter; Preul, Mark C.; Yang, Yezhou.

In: Journal of Visual Communication and Image Representation, Vol. 54, 01.07.2018, p. 10-20.

Research output: Contribution to journalArticle

Izadyyazdanabadi, Mohammadhassan ; Belykh, Evgenii ; Mooney, Michael ; Martirosyan, Nikolay ; Eschbacher, Jennifer ; Nakaji, Peter ; Preul, Mark C. ; Yang, Yezhou. / Convolutional neural networks : Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images. In: Journal of Visual Communication and Image Representation. 2018 ; Vol. 54. pp. 10-20.
@article{6abd54130b3748c6a49e858d04bd9322,
title = "Convolutional neural networks: Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images",
abstract = "Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence technology undergoing assessment for applications in brain tumor surgery. Many of the CLE images can be distorted and interpreted as nondiagnostic. However, just one neat CLE image might suffice for intraoperative diagnosis of the tumor. While manual examination of thousands of nondiagnostic images during surgery would be impractical, this creates an opportunity for a model to select diagnostic images for the pathologists or surgeons review. In this study, we sought to develop a deep learning model to automatically detect the diagnostic images. We explored the effect of training regimes and ensemble modeling and localized histological features from diagnostic CLE images. The developed model could achieve promising agreement with the ground truth. With the speed and precision of the proposed method, it has potential to be integrated into the operative workflow in the brain tumor surgery.",
keywords = "Brain tumor, Confocal laser endomicroscopy, Ensemble modeling, Neural network, Surgical vision, Unsupervised localization",
author = "Mohammadhassan Izadyyazdanabadi and Evgenii Belykh and Michael Mooney and Nikolay Martirosyan and Jennifer Eschbacher and Peter Nakaji and Preul, {Mark C.} and Yezhou Yang",
year = "2018",
month = "7",
day = "1",
doi = "10.1016/j.jvcir.2018.04.004",
language = "English (US)",
volume = "54",
pages = "10--20",
journal = "Journal of Visual Communication and Image Representation",
issn = "1047-3203",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Convolutional neural networks

T2 - Ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical CLE images

AU - Izadyyazdanabadi, Mohammadhassan

AU - Belykh, Evgenii

AU - Mooney, Michael

AU - Martirosyan, Nikolay

AU - Eschbacher, Jennifer

AU - Nakaji, Peter

AU - Preul, Mark C.

AU - Yang, Yezhou

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence technology undergoing assessment for applications in brain tumor surgery. Many of the CLE images can be distorted and interpreted as nondiagnostic. However, just one neat CLE image might suffice for intraoperative diagnosis of the tumor. While manual examination of thousands of nondiagnostic images during surgery would be impractical, this creates an opportunity for a model to select diagnostic images for the pathologists or surgeons review. In this study, we sought to develop a deep learning model to automatically detect the diagnostic images. We explored the effect of training regimes and ensemble modeling and localized histological features from diagnostic CLE images. The developed model could achieve promising agreement with the ground truth. With the speed and precision of the proposed method, it has potential to be integrated into the operative workflow in the brain tumor surgery.

AB - Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence technology undergoing assessment for applications in brain tumor surgery. Many of the CLE images can be distorted and interpreted as nondiagnostic. However, just one neat CLE image might suffice for intraoperative diagnosis of the tumor. While manual examination of thousands of nondiagnostic images during surgery would be impractical, this creates an opportunity for a model to select diagnostic images for the pathologists or surgeons review. In this study, we sought to develop a deep learning model to automatically detect the diagnostic images. We explored the effect of training regimes and ensemble modeling and localized histological features from diagnostic CLE images. The developed model could achieve promising agreement with the ground truth. With the speed and precision of the proposed method, it has potential to be integrated into the operative workflow in the brain tumor surgery.

KW - Brain tumor

KW - Confocal laser endomicroscopy

KW - Ensemble modeling

KW - Neural network

KW - Surgical vision

KW - Unsupervised localization

UR - http://www.scopus.com/inward/record.url?scp=85046376836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046376836&partnerID=8YFLogxK

U2 - 10.1016/j.jvcir.2018.04.004

DO - 10.1016/j.jvcir.2018.04.004

M3 - Article

VL - 54

SP - 10

EP - 20

JO - Journal of Visual Communication and Image Representation

JF - Journal of Visual Communication and Image Representation

SN - 1047-3203

ER -