Deep Multimodal Brain Network Learning for Joint Analysis of Structural Morphometry and Functional Connectivity

Wen Zhang, Yalin Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Learning from the multimodal brain imaging data attracts a large amount of attention in medical image analysis due to the proliferation of multimodal data collection. It is widely accepted that multimodal data can provide complementary information than mining from a single modality. However, unifying the image-based knowledge from the multimodal data is very challenging due to different image signals, resolution, data structure, etc.. In this study, we design a supervised deep model to jointly analyze brain morphometry and functional connectivity on the cortical surface and we name it deep multimodal brain network learning (DMBNL). Two graph-based kernels, i.e., geometry-aware surface kernel (GSK) and topology-aware network kernel (TNK), are proposed for processing the cortical surface morphometry and brain functional network. The vertex features on the cortical surface from GSK is pooled and feed into TNK as its initial regional features. In the end, the graph-level feature is computed for each individual and thus can be applied for classification tasks. We test our model on a large autism imaging dataset. The experimental results prove the effectiveness of our model.

Original languageEnglish (US)
Title of host publicationISBI 2020 - 2020 IEEE International Symposium on Biomedical Imaging
PublisherIEEE Computer Society
Pages1924-1928
Number of pages5
ISBN (Electronic)9781538693308
DOIs
StatePublished - Apr 2020
Event17th IEEE International Symposium on Biomedical Imaging, ISBI 2020 - Iowa City, United States
Duration: Apr 3 2020Apr 7 2020

Publication series

NameProceedings - International Symposium on Biomedical Imaging
Volume2020-April
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452

Conference

Conference17th IEEE International Symposium on Biomedical Imaging, ISBI 2020
CountryUnited States
CityIowa City
Period4/3/204/7/20

Keywords

  • Multimodal fusion
  • brain cortical surface
  • deep learning
  • functional connectivity
  • graph

ASJC Scopus subject areas

  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging

Fingerprint Dive into the research topics of 'Deep Multimodal Brain Network Learning for Joint Analysis of Structural Morphometry and Functional Connectivity'. Together they form a unique fingerprint.

  • Cite this

    Zhang, W., & Wang, Y. (2020). Deep Multimodal Brain Network Learning for Joint Analysis of Structural Morphometry and Functional Connectivity. In ISBI 2020 - 2020 IEEE International Symposium on Biomedical Imaging (pp. 1924-1928). [9098624] (Proceedings - International Symposium on Biomedical Imaging; Vol. 2020-April). IEEE Computer Society. https://doi.org/10.1109/ISBI45749.2020.9098624