Abstract
We consider iterative algorithms of the form x:=f(x), executed by a parallel or distributed computing system. We first consider synchronous executions of such iterations and study their communication requirements, as well as issues related to processor synchronization. We also discuss the parallelization of iterations of the Gauss-Seidel type. We then consider asynchronous implementations whereby each processor iterates on a different component of x, at its own pace, using the most recently received (but possibly outdated) information on the remaining components of x. While certain algorithms may fail to converge when implemented asynchronously, a large number of positive convergence results is available. We classify asynchronous algorithms into three main categories, depending on the amount of asynchronism they can tolerate, and survey the corresponding convergence results. We also discuss issues related to their termination.
Original language | English (US) |
---|---|
Pages (from-to) | 3-21 |
Number of pages | 19 |
Journal | Automatica |
Volume | 27 |
Issue number | 1 |
DOIs | |
State | Published - Jan 1991 |
Externally published | Yes |
Keywords
- Computational methods
- asynchronous algorithms
- distributed algorithms
- distributed data processing
- iterative methods
- parallel algorithms
- parallel processing
ASJC Scopus subject areas
- Control and Systems Engineering
- Electrical and Electronic Engineering