Multi-stage Dantzig selector

Ji Liu, Peter Wonka, Jieping Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

We consider the following sparse signal recovery (or feature selection) problem: given a design matrix X ∈ ℝ n×m (m ≫ n) and a noisy observation vector y ∈ ℝ n satisfying y = Xβ* +ε where ε is the noise vector following a Gaussian distribution N(0; σ 2I), how to recover the signal (or parameter vector) β* when the signal is sparse? The Dantzig selector has been proposed for sparse signal recovery with strong theoretical guarantees. In this paper, we propose a multi-stage Dantzig selector method, which iteratively refines the target signal β* We show that if X obeys a certain condition, then with a large probability the difference between the solution β̂ estimated by the proposed method and the true solution β* measured in terms of the l p norm (p ≥ 1) is bounded as ∥ β̂- β* p≤ (C(s - N) 1/p √ logm + Δ)σ where C is a constant, s is the number of nonzero entries in β* Δ is independent of m and is much smaller than the first term, and N is the number of entries of β* larger than a certain value in the order of O(σ √ logm). The proposed method improves the estimation bound of the standard Dantzig selector approximately from Cs 1/p √ logmσ to C(s-N) 1/p √ logmσ where the value N depends on the number of large entries in β* When N = s, the proposed algorithm achieves the oracle solution with a high probability. In addition, with a large probability, the proposed method can select the same number of correct features under a milder condition than the Dantzig selector.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 23
Subtitle of host publication24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
StatePublished - Dec 1 2010
Event24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 - Vancouver, BC, Canada
Duration: Dec 6 2010Dec 9 2010

Publication series

NameAdvances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Other

Other24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
CountryCanada
CityVancouver, BC
Period12/6/1012/9/10

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Multi-stage Dantzig selector'. Together they form a unique fingerprint.

  • Cite this

    Liu, J., Wonka, P., & Ye, J. (2010). Multi-stage Dantzig selector. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 (Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010).