TY - GEN
T1 - Deep Multimodal Brain Network Learning for Joint Analysis of Structural Morphometry and Functional Connectivity
AU - Zhang, Wen
AU - Wang, Yalin
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/4
Y1 - 2020/4
N2 - Learning from the multimodal brain imaging data attracts a large amount of attention in medical image analysis due to the proliferation of multimodal data collection. It is widely accepted that multimodal data can provide complementary information than mining from a single modality. However, unifying the image-based knowledge from the multimodal data is very challenging due to different image signals, resolution, data structure, etc.. In this study, we design a supervised deep model to jointly analyze brain morphometry and functional connectivity on the cortical surface and we name it deep multimodal brain network learning (DMBNL). Two graph-based kernels, i.e., geometry-aware surface kernel (GSK) and topology-aware network kernel (TNK), are proposed for processing the cortical surface morphometry and brain functional network. The vertex features on the cortical surface from GSK is pooled and feed into TNK as its initial regional features. In the end, the graph-level feature is computed for each individual and thus can be applied for classification tasks. We test our model on a large autism imaging dataset. The experimental results prove the effectiveness of our model.
AB - Learning from the multimodal brain imaging data attracts a large amount of attention in medical image analysis due to the proliferation of multimodal data collection. It is widely accepted that multimodal data can provide complementary information than mining from a single modality. However, unifying the image-based knowledge from the multimodal data is very challenging due to different image signals, resolution, data structure, etc.. In this study, we design a supervised deep model to jointly analyze brain morphometry and functional connectivity on the cortical surface and we name it deep multimodal brain network learning (DMBNL). Two graph-based kernels, i.e., geometry-aware surface kernel (GSK) and topology-aware network kernel (TNK), are proposed for processing the cortical surface morphometry and brain functional network. The vertex features on the cortical surface from GSK is pooled and feed into TNK as its initial regional features. In the end, the graph-level feature is computed for each individual and thus can be applied for classification tasks. We test our model on a large autism imaging dataset. The experimental results prove the effectiveness of our model.
KW - Multimodal fusion
KW - brain cortical surface
KW - deep learning
KW - functional connectivity
KW - graph
UR - http://www.scopus.com/inward/record.url?scp=85085857403&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085857403&partnerID=8YFLogxK
U2 - 10.1109/ISBI45749.2020.9098624
DO - 10.1109/ISBI45749.2020.9098624
M3 - Conference contribution
AN - SCOPUS:85085857403
T3 - Proceedings - International Symposium on Biomedical Imaging
SP - 1924
EP - 1928
BT - ISBI 2020 - 2020 IEEE International Symposium on Biomedical Imaging
PB - IEEE Computer Society
T2 - 17th IEEE International Symposium on Biomedical Imaging, ISBI 2020
Y2 - 3 April 2020 through 7 April 2020
ER -