A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

Hoi To Wai, Tsung Hui Chang, Anna Scaglione

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

In handling massive-scale signal processing problems arising from 'big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensus-based methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including 'sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.

Original languageEnglish (US)
Title of host publicationICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3546-3550
Number of pages5
Volume2015-August
ISBN (Print)9781467369978
DOIs
StatePublished - Aug 4 2015
Event40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Brisbane, Australia
Duration: Apr 19 2014Apr 24 2014

Other

Other40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015
CountryAustralia
CityBrisbane
Period4/19/144/24/14

Fingerprint

Glossaries
Fault tolerance
Signal processing

Keywords

  • decentralized algorithm
  • dictionary learning
  • non-convex optimization

ASJC Scopus subject areas

  • Signal Processing
  • Software
  • Electrical and Electronic Engineering

Cite this

Wai, H. T., Chang, T. H., & Scaglione, A. (2015). A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (Vol. 2015-August, pp. 3546-3550). [7178631] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2015.7178631

A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning. / Wai, Hoi To; Chang, Tsung Hui; Scaglione, Anna.

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Vol. 2015-August Institute of Electrical and Electronics Engineers Inc., 2015. p. 3546-3550 7178631.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wai, HT, Chang, TH & Scaglione, A 2015, A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning. in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. vol. 2015-August, 7178631, Institute of Electrical and Electronics Engineers Inc., pp. 3546-3550, 40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015, Brisbane, Australia, 4/19/14. https://doi.org/10.1109/ICASSP.2015.7178631
Wai HT, Chang TH, Scaglione A. A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Vol. 2015-August. Institute of Electrical and Electronics Engineers Inc. 2015. p. 3546-3550. 7178631 https://doi.org/10.1109/ICASSP.2015.7178631
Wai, Hoi To ; Chang, Tsung Hui ; Scaglione, Anna. / A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. Vol. 2015-August Institute of Electrical and Electronics Engineers Inc., 2015. pp. 3546-3550
@inproceedings{50f6c856d81141248b5ee7a25150ea7d,
title = "A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning",
abstract = "In handling massive-scale signal processing problems arising from 'big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensus-based methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including 'sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.",
keywords = "decentralized algorithm, dictionary learning, non-convex optimization",
author = "Wai, {Hoi To} and Chang, {Tsung Hui} and Anna Scaglione",
year = "2015",
month = "8",
day = "4",
doi = "10.1109/ICASSP.2015.7178631",
language = "English (US)",
isbn = "9781467369978",
volume = "2015-August",
pages = "3546--3550",
booktitle = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

AU - Wai, Hoi To

AU - Chang, Tsung Hui

AU - Scaglione, Anna

PY - 2015/8/4

Y1 - 2015/8/4

N2 - In handling massive-scale signal processing problems arising from 'big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensus-based methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including 'sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.

AB - In handling massive-scale signal processing problems arising from 'big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensus-based methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including 'sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.

KW - decentralized algorithm

KW - dictionary learning

KW - non-convex optimization

UR - http://www.scopus.com/inward/record.url?scp=84946016206&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84946016206&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2015.7178631

DO - 10.1109/ICASSP.2015.7178631

M3 - Conference contribution

AN - SCOPUS:84946016206

SN - 9781467369978

VL - 2015-August

SP - 3546

EP - 3550

BT - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -