A Geometric Framework for Feature Mappings in Multimodal Fusion of Brain Image Data

Wen Zhang, Liang Mi, Paul M. Thompson, Yalin Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Fusing multimodal brain image features to empower statistical analysis has attracted considerable research interest. Generally, a feature mapping is learned in the fusion process so the cross-modality relationship in the multimodal data can be more effectively extracted in a common feature space. Most of the prior work achieve this goal by data-driven approaches without considering the geometry properties of the feature spaces where the data are embedded. It results in a huge sacrifice of untapped information. Here, we propose to fuse the multimodal brain images through a novel geometric approach. The key idea is to encode various brain image features with the local metric change on brain shapes, such that the feature mapping can be efficiently solved by some geometric mapping functions, i.e., quasiconformal and harmonic mappings. We approach our multimodal fusion framework (MFRM) in two steps: surface feature mapping and volumetric feature mapping. For each of them, we design an informative Riemannian metric based on distinct brain anatomical features and achieve image fusion via diffeomorphic maps. We evaluate our proposed method on two brain image cohorts. The experimental results reveal the effectiveness of our proposed framework which yields better statistical performances than state-of-the-art data-driven methods.

Original languageEnglish (US)
Title of host publicationInformation Processing in Medical Imaging - 26th International Conference, IPMI 2019, Proceedings
EditorsSiqi Bao, Albert C.S. Chung, James C. Gee, Paul A. Yushkevich
PublisherSpringer Verlag
Pages617-630
Number of pages14
ISBN (Print)9783030203504
DOIs
StatePublished - 2019
Event26th International Conference on Information Processing in Medical Imaging, IPMI 2019 - Hong Kong, China
Duration: Jun 2 2019Jun 7 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11492 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference26th International Conference on Information Processing in Medical Imaging, IPMI 2019
CountryChina
CityHong Kong
Period6/2/196/7/19

Keywords

  • Harmonic mapping
  • Multimodal fusion
  • Quasiconformal mapping
  • Riemannian metric
  • diffusion MRI
  • structural MRI

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'A Geometric Framework for Feature Mappings in Multimodal Fusion of Brain Image Data'. Together they form a unique fingerprint.

Cite this