On stochastic proximal-point method for convex-composite optimization

Angelia Nedich, Tatiana Tatarenko

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We study stochastic proximal-point method applied to a convex-composite optimization problem, where the objective function is given as the sum of two convex functions, one of which is smooth while the other is not necessarily smooth but has a simple structure for evaluating the proximal operator. The main goal is to investigate a trade-off between the choice of a constant stepsize value and the speed at which the algorithm approaches the optimal points. We consider the case of a strongly convex objective function and make the most standard assumptions on the smooth component function and its stochastic gradient estimates. First of all, we analyze the basic properties of the stochastic proximal-point mapping associated with the procedure under consideration. Based on these properties, we formulate the main result, which provides the explicit condition on the constant stepsize for which the stochastic proximal-point method approaches a σ-neighborhood of the optimal point in expectation, where the parameter σ > 0 is related to the variance of the stochastic gradient estimates. Moreover, the rate at which the σ-neighborhood attracts the iterates is geometric, which allows us to estimate the number of iterations the procedure needs to enter this region (in expectation).

Original languageEnglish (US)
Title of host publication55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages198-203
Number of pages6
Volume2018-January
ISBN (Electronic)9781538632666
DOIs
StatePublished - Jan 17 2018
Event55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017 - Monticello, United States
Duration: Oct 3 2017Oct 6 2017

Other

Other55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
CountryUnited States
CityMonticello
Period10/3/1710/6/17

Fingerprint

Proximal Point Method
Stochastic Gradient
Gradient Estimate
Composite
Convex function
Optimization
Composite materials
Objective function
Proximal Point
Iterate
Trade-offs
Optimization Problem
Iteration
Operator
Estimate

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Energy Engineering and Power Technology
  • Control and Optimization

Cite this

Nedich, A., & Tatarenko, T. (2018). On stochastic proximal-point method for convex-composite optimization. In 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017 (Vol. 2018-January, pp. 198-203). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ALLERTON.2017.8262738

On stochastic proximal-point method for convex-composite optimization. / Nedich, Angelia; Tatarenko, Tatiana.

55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 198-203.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nedich, A & Tatarenko, T 2018, On stochastic proximal-point method for convex-composite optimization. in 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 198-203, 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017, Monticello, United States, 10/3/17. https://doi.org/10.1109/ALLERTON.2017.8262738
Nedich A, Tatarenko T. On stochastic proximal-point method for convex-composite optimization. In 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 198-203 https://doi.org/10.1109/ALLERTON.2017.8262738
Nedich, Angelia ; Tatarenko, Tatiana. / On stochastic proximal-point method for convex-composite optimization. 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 198-203
@inproceedings{9de77984f4174e85b8d5728f2dd6ac5b,
title = "On stochastic proximal-point method for convex-composite optimization",
abstract = "We study stochastic proximal-point method applied to a convex-composite optimization problem, where the objective function is given as the sum of two convex functions, one of which is smooth while the other is not necessarily smooth but has a simple structure for evaluating the proximal operator. The main goal is to investigate a trade-off between the choice of a constant stepsize value and the speed at which the algorithm approaches the optimal points. We consider the case of a strongly convex objective function and make the most standard assumptions on the smooth component function and its stochastic gradient estimates. First of all, we analyze the basic properties of the stochastic proximal-point mapping associated with the procedure under consideration. Based on these properties, we formulate the main result, which provides the explicit condition on the constant stepsize for which the stochastic proximal-point method approaches a σ-neighborhood of the optimal point in expectation, where the parameter σ > 0 is related to the variance of the stochastic gradient estimates. Moreover, the rate at which the σ-neighborhood attracts the iterates is geometric, which allows us to estimate the number of iterations the procedure needs to enter this region (in expectation).",
author = "Angelia Nedich and Tatiana Tatarenko",
year = "2018",
month = "1",
day = "17",
doi = "10.1109/ALLERTON.2017.8262738",
language = "English (US)",
volume = "2018-January",
pages = "198--203",
booktitle = "55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - On stochastic proximal-point method for convex-composite optimization

AU - Nedich, Angelia

AU - Tatarenko, Tatiana

PY - 2018/1/17

Y1 - 2018/1/17

N2 - We study stochastic proximal-point method applied to a convex-composite optimization problem, where the objective function is given as the sum of two convex functions, one of which is smooth while the other is not necessarily smooth but has a simple structure for evaluating the proximal operator. The main goal is to investigate a trade-off between the choice of a constant stepsize value and the speed at which the algorithm approaches the optimal points. We consider the case of a strongly convex objective function and make the most standard assumptions on the smooth component function and its stochastic gradient estimates. First of all, we analyze the basic properties of the stochastic proximal-point mapping associated with the procedure under consideration. Based on these properties, we formulate the main result, which provides the explicit condition on the constant stepsize for which the stochastic proximal-point method approaches a σ-neighborhood of the optimal point in expectation, where the parameter σ > 0 is related to the variance of the stochastic gradient estimates. Moreover, the rate at which the σ-neighborhood attracts the iterates is geometric, which allows us to estimate the number of iterations the procedure needs to enter this region (in expectation).

AB - We study stochastic proximal-point method applied to a convex-composite optimization problem, where the objective function is given as the sum of two convex functions, one of which is smooth while the other is not necessarily smooth but has a simple structure for evaluating the proximal operator. The main goal is to investigate a trade-off between the choice of a constant stepsize value and the speed at which the algorithm approaches the optimal points. We consider the case of a strongly convex objective function and make the most standard assumptions on the smooth component function and its stochastic gradient estimates. First of all, we analyze the basic properties of the stochastic proximal-point mapping associated with the procedure under consideration. Based on these properties, we formulate the main result, which provides the explicit condition on the constant stepsize for which the stochastic proximal-point method approaches a σ-neighborhood of the optimal point in expectation, where the parameter σ > 0 is related to the variance of the stochastic gradient estimates. Moreover, the rate at which the σ-neighborhood attracts the iterates is geometric, which allows us to estimate the number of iterations the procedure needs to enter this region (in expectation).

UR - http://www.scopus.com/inward/record.url?scp=85047993686&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047993686&partnerID=8YFLogxK

U2 - 10.1109/ALLERTON.2017.8262738

DO - 10.1109/ALLERTON.2017.8262738

M3 - Conference contribution

VL - 2018-January

SP - 198

EP - 203

BT - 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -