Proximal algorithms and temporal difference methods for solving fixed point problems

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

In this paper we consider large fixed point problems and solution with proximal algorithms. We show that for linear problems there is a close connection between proximal iterations, which are prominent in numerical analysis and optimization, and multistep methods of the temporal difference type such as TD(λ), LSTD(λ), and LSPE(λ), which are central in simulation-based exact and approximate dynamic programming. One benefit of this connection is a new and simple way to accelerate the standard proximal algorithm by extrapolation towards a multistep iteration, which generically has a faster convergence rate. Another benefit is the potential for integration into the proximal algorithmic context of several new ideas that have emerged in the approximate dynamic programming context, including simulation-based implementations. Conversely, the analytical and algorithmic insights from proximal algorithms can be brought to bear on the analysis and the enhancement of temporal difference methods. We also generalize our linear case result to nonlinear problems that involve a contractive mapping, thus providing guaranteed and potentially substantial acceleration of the proximal and forward backward splitting algorithms at no extra cost. Moreover, under certain monotonicity assumptions, we extend the connection with temporal difference methods to nonlinear problems through a linearization approach.

Original languageEnglish (US)
Pages (from-to)709-736
Number of pages28
JournalComputational Optimization and Applications
Volume70
Issue number3
DOIs
StatePublished - Jul 1 2018
Externally publishedYes

Keywords

  • Convex optimization
  • Dynamic programming
  • Fixed point problems
  • Proximal algorithm
  • Temporal differences

ASJC Scopus subject areas

  • Control and Optimization
  • Computational Mathematics
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Proximal algorithms and temporal difference methods for solving fixed point problems'. Together they form a unique fingerprint.

Cite this