Recovering non-negative and combined sparse representations

Karthikeyan Natesan Ramamurthy, Jayaraman J. Thiagarajan, Andreas Spanias

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the coefficient vector is constrained to be non-negative, and the rest is unconstrained (general). We analyze the recovery of the unique, sparsest solution, for combined representations, under three different cases of coefficient support knowledge: (a) the non-zero supports of non-negative and general coefficients are known, (b) the non-zero support of general coefficients alone is known, and (c) both the non-zero supports are unknown. For case (c), we propose the combined orthogonal matching pursuit algorithm for coefficient recovery and derive the deterministic sparsity threshold under which recovery of the unique, sparsest coefficient vector is possible. We quantify the order complexity of the algorithms, and examine their performance in exact and approximate recovery of coefficients under various conditions of noise. Furthermore, we also obtain their empirical phase transition characteristics. We show that the basis pursuit algorithm, with partial non-negative constraints, and the proposed greedy algorithm perform better in recovering the unique sparse representation when compared to their unconstrained counterparts. Finally, we demonstrate the utility of the proposed methods in recovering images corrupted by saturation noise.

Original languageEnglish (US)
Pages (from-to)21-35
Number of pages15
JournalDigital Signal Processing: A Review Journal
Volume26
Issue number1
DOIs
StatePublished - Mar 2014

Fingerprint

Recovery
Linear systems
Phase transitions

Keywords

  • Non-negative representations
  • Orthogonal matching pursuit
  • Sparse representations
  • Underdetermined linear system
  • Unique sparse solution

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Recovering non-negative and combined sparse representations. / Ramamurthy, Karthikeyan Natesan; Thiagarajan, Jayaraman J.; Spanias, Andreas.

In: Digital Signal Processing: A Review Journal, Vol. 26, No. 1, 03.2014, p. 21-35.

Research output: Contribution to journalArticle

Ramamurthy, Karthikeyan Natesan ; Thiagarajan, Jayaraman J. ; Spanias, Andreas. / Recovering non-negative and combined sparse representations. In: Digital Signal Processing: A Review Journal. 2014 ; Vol. 26, No. 1. pp. 21-35.
@article{f670987d418648098ba870002c891498,
title = "Recovering non-negative and combined sparse representations",
abstract = "The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the coefficient vector is constrained to be non-negative, and the rest is unconstrained (general). We analyze the recovery of the unique, sparsest solution, for combined representations, under three different cases of coefficient support knowledge: (a) the non-zero supports of non-negative and general coefficients are known, (b) the non-zero support of general coefficients alone is known, and (c) both the non-zero supports are unknown. For case (c), we propose the combined orthogonal matching pursuit algorithm for coefficient recovery and derive the deterministic sparsity threshold under which recovery of the unique, sparsest coefficient vector is possible. We quantify the order complexity of the algorithms, and examine their performance in exact and approximate recovery of coefficients under various conditions of noise. Furthermore, we also obtain their empirical phase transition characteristics. We show that the basis pursuit algorithm, with partial non-negative constraints, and the proposed greedy algorithm perform better in recovering the unique sparse representation when compared to their unconstrained counterparts. Finally, we demonstrate the utility of the proposed methods in recovering images corrupted by saturation noise.",
keywords = "Non-negative representations, Orthogonal matching pursuit, Sparse representations, Underdetermined linear system, Unique sparse solution",
author = "Ramamurthy, {Karthikeyan Natesan} and Thiagarajan, {Jayaraman J.} and Andreas Spanias",
year = "2014",
month = "3",
doi = "10.1016/j.dsp.2013.11.003",
language = "English (US)",
volume = "26",
pages = "21--35",
journal = "Digital Signal Processing: A Review Journal",
issn = "1051-2004",
publisher = "Elsevier Inc.",
number = "1",

}

TY - JOUR

T1 - Recovering non-negative and combined sparse representations

AU - Ramamurthy, Karthikeyan Natesan

AU - Thiagarajan, Jayaraman J.

AU - Spanias, Andreas

PY - 2014/3

Y1 - 2014/3

N2 - The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the coefficient vector is constrained to be non-negative, and the rest is unconstrained (general). We analyze the recovery of the unique, sparsest solution, for combined representations, under three different cases of coefficient support knowledge: (a) the non-zero supports of non-negative and general coefficients are known, (b) the non-zero support of general coefficients alone is known, and (c) both the non-zero supports are unknown. For case (c), we propose the combined orthogonal matching pursuit algorithm for coefficient recovery and derive the deterministic sparsity threshold under which recovery of the unique, sparsest coefficient vector is possible. We quantify the order complexity of the algorithms, and examine their performance in exact and approximate recovery of coefficients under various conditions of noise. Furthermore, we also obtain their empirical phase transition characteristics. We show that the basis pursuit algorithm, with partial non-negative constraints, and the proposed greedy algorithm perform better in recovering the unique sparse representation when compared to their unconstrained counterparts. Finally, we demonstrate the utility of the proposed methods in recovering images corrupted by saturation noise.

AB - The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the coefficient vector is constrained to be non-negative, and the rest is unconstrained (general). We analyze the recovery of the unique, sparsest solution, for combined representations, under three different cases of coefficient support knowledge: (a) the non-zero supports of non-negative and general coefficients are known, (b) the non-zero support of general coefficients alone is known, and (c) both the non-zero supports are unknown. For case (c), we propose the combined orthogonal matching pursuit algorithm for coefficient recovery and derive the deterministic sparsity threshold under which recovery of the unique, sparsest coefficient vector is possible. We quantify the order complexity of the algorithms, and examine their performance in exact and approximate recovery of coefficients under various conditions of noise. Furthermore, we also obtain their empirical phase transition characteristics. We show that the basis pursuit algorithm, with partial non-negative constraints, and the proposed greedy algorithm perform better in recovering the unique sparse representation when compared to their unconstrained counterparts. Finally, we demonstrate the utility of the proposed methods in recovering images corrupted by saturation noise.

KW - Non-negative representations

KW - Orthogonal matching pursuit

KW - Sparse representations

KW - Underdetermined linear system

KW - Unique sparse solution

UR - http://www.scopus.com/inward/record.url?scp=84893968733&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893968733&partnerID=8YFLogxK

U2 - 10.1016/j.dsp.2013.11.003

DO - 10.1016/j.dsp.2013.11.003

M3 - Article

AN - SCOPUS:84893968733

VL - 26

SP - 21

EP - 35

JO - Digital Signal Processing: A Review Journal

JF - Digital Signal Processing: A Review Journal

SN - 1051-2004

IS - 1

ER -