The illusion of the illusion of sparsity: An exercise in prior sensitivity

Bruno Fava, Hedibert F. Lopes

Research output: Contribution to journalArticlepeer-review

Abstract

The emergence of Big Data raises the question of how to model economic relations when there is a large number of possible explanatory variables. We revisit the issue by comparing the possibility of using dense or sparse models in a Bayesian approach, allowing for variable selection and shrinkage. More specifically, we discuss the results reached by Giannone, Lenza and Primiceri (2020) through a “Spike-and-Slab” prior, which suggest an “illusion of sparsity” in economic data, as no clear patterns of sparsity could be detected. We make a further revision of the posterior distributions of the model, and propose three experiments to evaluate the robustness of the adopted prior distribution. We find that the pattern of sparsity is sensitive to the prior distribution of the regression coefficients, and present evidence that the model indirectly induces variable selection and shrinkage, which suggests that the “illusion of sparsity” could be, itself, an illusion. Code is available on Github.

Original languageEnglish (US)
Pages (from-to)699-720
Number of pages22
JournalBrazilian Journal of Probability and Statistics
Volume35
Issue number4
DOIs
StatePublished - Nov 2021
Externally publishedYes

Keywords

  • Bayesian econometrics
  • High dimensional data
  • Model selection
  • Shrinkage
  • Sparsity

ASJC Scopus subject areas

  • Statistics and Probability

Fingerprint

Dive into the research topics of 'The illusion of the illusion of sparsity: An exercise in prior sensitivity'. Together they form a unique fingerprint.

Cite this