Analysis of sampling techniques for imbalanced data: An n=648 ADNI study

Rashmi Dubey, Jiayu Zhou, Yalin Wang, Paul M. Thompson, Jieping Ye

Research output: Contribution to journalArticle

81 Scopus citations

Abstract

Many neuroimaging applications deal with imbalanced imaging data. For example, in Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, the mild cognitive impairment (MCI) cases eligible for the study are nearly two times the Alzheimer's disease (AD) patients for structural magnetic resonance imaging (MRI) modality and six times the control cases for proteomics modality. Constructing an accurate classifier from imbalanced data is a challenging task. Traditional classifiers that aim to maximize the overall prediction accuracy tend to classify all data into the majority class. In this paper, we study an ensemble system of feature selection and data sampling for the class imbalance problem. We systematically analyze various sampling techniques by examining the efficacy of different rates and types of undersampling, oversampling, and a combination of over and undersampling approaches. We thoroughly examine six widely used feature selection algorithms to identify significant biomarkers and thereby reduce the complexity of the data. The efficacy of the ensemble techniques is evaluated using two different classifiers including Random Forest and Support Vector Machines based on classification accuracy, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity measures. Our extensive experimental results show that for various problem settings in ADNI, (1) a balanced training set obtained with K-Medoids technique based undersampling gives the best overall performance among different data sampling techniques and no sampling approach; and (2) sparse logistic regression with stability selection achieves competitive performance among various feature selection algorithms. Comprehensive experiments with various settings show that our proposed ensemble model of multiple undersampled datasets yields stable and promising results.

Original languageEnglish (US)
Pages (from-to)220-241
Number of pages22
JournalNeuroImage
Volume87
DOIs
StatePublished - Feb 15 2014

    Fingerprint

Keywords

  • Alzheimer's disease
  • Classification
  • Feature selection
  • Imbalanced data
  • Oversampling
  • Undersampling

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience

Cite this