Resampling methods for variable selection in robust regression

James W. Wisnowski, James R. Simpson, Douglas Montgomery, George Runger

Research output: Contribution to journalArticle

25 Scopus citations

Abstract

With the inundation of large data sets requiring analysis and empirical model building, outliers have become commonplace. Fortunately, several standard statistical software packages have allowed practitioners to use robust regression estimators to easily fit data sets that are contaminated with outliers. However, little guidance is available for selecting the best subset of the predictor variables when using these robust estimators. We initially consider cross-validation and bootstrap resampling methods that have performed well for least-squares variable selection. It turns out that these variable selection methods cannot be directly applied to contaminated data sets using a robust estimation scheme. The prediction errors, inflated by the outliers, are not reliable measures of how well the robust model fits the data. As a result, new resampling variable selection methods are proposed by introducing alternative estimates of prediction error in the contaminated model. We demonstrate that, although robust estimation and resampling variable selection are computationally complex procedures, we can combine both techniques for superior results using modest computational resources. Monte Carlo simulation is used to evaluate the proposed variable selection procedures against alternatives through a designed experiment approach. The experiment factors include percentage of outliers, outlier geometry, bootstrap sample size, number of bootstrap samples, and cross-validation assessment size. The results are summarized and recommendations for use are provided.

Original languageEnglish (US)
Pages (from-to)341-355
Number of pages15
JournalComputational Statistics and Data Analysis
Volume43
Issue number3
DOIs
StatePublished - Jul 28 2003

    Fingerprint

Keywords

  • Bootstrap
  • Cross-validation
  • Outliers
  • Robust regression
  • Variable selection

ASJC Scopus subject areas

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this