A projection-free decentralized algorithm for non-convex optimization

Hoi To Wai, Anna Scaglione, Jean Lafond, Eric Moulines

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

This paper considers a decentralized projection free algorithm for non-convex optimization in high dimension. More specifically, we propose a Decentralized Frank-Wolfe (DeFW) algorithm which is suitable when high dimensional optimization constraints are difficult to handle by conventional projection/proximal-based gradient descent methods. We present conditions under which the DeFW algorithm converges to a stationary point and prove that the rate of convergence is as fast as O(l/√T), where T is the iteration number. This paper provides the first convergence guarantee for FrankWolfe methods applied to non-convex decentralized optimization. Utilizing our theoretical findings, we formulate a novel robust matrix completion problem and apply DeFW to give an efficient decentralized solution. Numerical experiments are performed on realistic and synthetic data to support our findings.

Original languageEnglish (US)
Title of host publication2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages475-479
Number of pages5
ISBN (Electronic)9781509045457
DOIs
StatePublished - Apr 19 2017
Event2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Washington, United States
Duration: Dec 7 2016Dec 9 2016

Other

Other2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016
CountryUnited States
CityWashington
Period12/7/1612/9/16

Fingerprint

Experiments

Keywords

  • Decentralized algorithms
  • Frank-Wolfe method
  • Gossip algorithms
  • Matrix completion
  • Non-convex optimization

ASJC Scopus subject areas

  • Signal Processing
  • Computer Networks and Communications

Cite this

Wai, H. T., Scaglione, A., Lafond, J., & Moulines, E. (2017). A projection-free decentralized algorithm for non-convex optimization. In 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings (pp. 475-479). [7905887] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GlobalSIP.2016.7905887

A projection-free decentralized algorithm for non-convex optimization. / Wai, Hoi To; Scaglione, Anna; Lafond, Jean; Moulines, Eric.

2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 475-479 7905887.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wai, HT, Scaglione, A, Lafond, J & Moulines, E 2017, A projection-free decentralized algorithm for non-convex optimization. in 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings., 7905887, Institute of Electrical and Electronics Engineers Inc., pp. 475-479, 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016, Washington, United States, 12/7/16. https://doi.org/10.1109/GlobalSIP.2016.7905887
Wai HT, Scaglione A, Lafond J, Moulines E. A projection-free decentralized algorithm for non-convex optimization. In 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 475-479. 7905887 https://doi.org/10.1109/GlobalSIP.2016.7905887
Wai, Hoi To ; Scaglione, Anna ; Lafond, Jean ; Moulines, Eric. / A projection-free decentralized algorithm for non-convex optimization. 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 475-479
@inproceedings{1afa352c1f2543a38c2881bf3fc1dba5,
title = "A projection-free decentralized algorithm for non-convex optimization",
abstract = "This paper considers a decentralized projection free algorithm for non-convex optimization in high dimension. More specifically, we propose a Decentralized Frank-Wolfe (DeFW) algorithm which is suitable when high dimensional optimization constraints are difficult to handle by conventional projection/proximal-based gradient descent methods. We present conditions under which the DeFW algorithm converges to a stationary point and prove that the rate of convergence is as fast as O(l/√T), where T is the iteration number. This paper provides the first convergence guarantee for FrankWolfe methods applied to non-convex decentralized optimization. Utilizing our theoretical findings, we formulate a novel robust matrix completion problem and apply DeFW to give an efficient decentralized solution. Numerical experiments are performed on realistic and synthetic data to support our findings.",
keywords = "Decentralized algorithms, Frank-Wolfe method, Gossip algorithms, Matrix completion, Non-convex optimization",
author = "Wai, {Hoi To} and Anna Scaglione and Jean Lafond and Eric Moulines",
year = "2017",
month = "4",
day = "19",
doi = "10.1109/GlobalSIP.2016.7905887",
language = "English (US)",
pages = "475--479",
booktitle = "2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - A projection-free decentralized algorithm for non-convex optimization

AU - Wai, Hoi To

AU - Scaglione, Anna

AU - Lafond, Jean

AU - Moulines, Eric

PY - 2017/4/19

Y1 - 2017/4/19

N2 - This paper considers a decentralized projection free algorithm for non-convex optimization in high dimension. More specifically, we propose a Decentralized Frank-Wolfe (DeFW) algorithm which is suitable when high dimensional optimization constraints are difficult to handle by conventional projection/proximal-based gradient descent methods. We present conditions under which the DeFW algorithm converges to a stationary point and prove that the rate of convergence is as fast as O(l/√T), where T is the iteration number. This paper provides the first convergence guarantee for FrankWolfe methods applied to non-convex decentralized optimization. Utilizing our theoretical findings, we formulate a novel robust matrix completion problem and apply DeFW to give an efficient decentralized solution. Numerical experiments are performed on realistic and synthetic data to support our findings.

AB - This paper considers a decentralized projection free algorithm for non-convex optimization in high dimension. More specifically, we propose a Decentralized Frank-Wolfe (DeFW) algorithm which is suitable when high dimensional optimization constraints are difficult to handle by conventional projection/proximal-based gradient descent methods. We present conditions under which the DeFW algorithm converges to a stationary point and prove that the rate of convergence is as fast as O(l/√T), where T is the iteration number. This paper provides the first convergence guarantee for FrankWolfe methods applied to non-convex decentralized optimization. Utilizing our theoretical findings, we formulate a novel robust matrix completion problem and apply DeFW to give an efficient decentralized solution. Numerical experiments are performed on realistic and synthetic data to support our findings.

KW - Decentralized algorithms

KW - Frank-Wolfe method

KW - Gossip algorithms

KW - Matrix completion

KW - Non-convex optimization

UR - http://www.scopus.com/inward/record.url?scp=85019197644&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019197644&partnerID=8YFLogxK

U2 - 10.1109/GlobalSIP.2016.7905887

DO - 10.1109/GlobalSIP.2016.7905887

M3 - Conference contribution

AN - SCOPUS:85019197644

SP - 475

EP - 479

BT - 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -