Efficient algorithms for solution of regularized total least squares

Rosemary Renaut, Hongbin Guo

Research output: Contribution to journalArticle

45 Citations (Scopus)

Abstract

Error-contaminated systems Ax ≈ b, for which A is ill-conditioned, are considered. Such systems may be solved using Tikhonov-like regularized total least squares (RTLS) methods. Golub, Hansen, and O'Leary [SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185-194] presented a parameter-dependent direct algorithm for the solution of the augmented Lagrange formulation for the RTLS problem, and Sima, Van Huffel, and Golub [Regularized Total Least Squares Based on Quadratic Eigenvalue Problem Solvers, Tech. Report SCCM-03-03, SCCM, Stanford University, Stanford, CA, 2003] have introduced a technique for solution based on a quadratic eigenvalue problem, RTLSQEP. Guo and Renaut [A regularized total least squares algorithm, in Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002, pp. 57-66] derived an eigenproblem for the RTLS which can be solved using the iterative inverse power method. Here we present an alternative derivation of the eigenproblem for constrained TLS through the augmented Lagrangian for the constrained normalized residual. This extends the analysis of the eigenproblem and leads to derivation of more efficient algorithms compared to the original formulation. Additional algorithms based on bisection search and a standard L-curve approach are presented. These algorithms vary with respect to the parameters that need to be prescribed. Numerical and convergence results supporting the different versions and contrasting with RTLSQEP are presented.

Original languageEnglish (US)
Pages (from-to)457-476
Number of pages20
JournalSIAM Journal on Matrix Analysis and Applications
Volume26
Issue number2
DOIs
StatePublished - 2005

Fingerprint

Total Least Squares
Efficient Algorithms
Eigenproblem
Quadratic Eigenvalue Problem
Augmented Lagrangians
L-curve
Errors in Variables
Power Method
Augmented Lagrangian
Algorithm Analysis
Inverse Method
Bisection
Formulation
Least Square Algorithm
Least Squares Problem
Least Square Method
Lagrange
Convergence Results
Vary
Numerical Results

Keywords

  • Ill-posedness
  • Rayleigh quotient iteration
  • Regularization
  • Total least squares

ASJC Scopus subject areas

  • Algebra and Number Theory
  • Applied Mathematics
  • Analysis

Cite this

Efficient algorithms for solution of regularized total least squares. / Renaut, Rosemary; Guo, Hongbin.

In: SIAM Journal on Matrix Analysis and Applications, Vol. 26, No. 2, 2005, p. 457-476.

Research output: Contribution to journalArticle

@article{41cba00724fd46fe94774bbbf05559f2,
title = "Efficient algorithms for solution of regularized total least squares",
abstract = "Error-contaminated systems Ax ≈ b, for which A is ill-conditioned, are considered. Such systems may be solved using Tikhonov-like regularized total least squares (RTLS) methods. Golub, Hansen, and O'Leary [SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185-194] presented a parameter-dependent direct algorithm for the solution of the augmented Lagrange formulation for the RTLS problem, and Sima, Van Huffel, and Golub [Regularized Total Least Squares Based on Quadratic Eigenvalue Problem Solvers, Tech. Report SCCM-03-03, SCCM, Stanford University, Stanford, CA, 2003] have introduced a technique for solution based on a quadratic eigenvalue problem, RTLSQEP. Guo and Renaut [A regularized total least squares algorithm, in Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002, pp. 57-66] derived an eigenproblem for the RTLS which can be solved using the iterative inverse power method. Here we present an alternative derivation of the eigenproblem for constrained TLS through the augmented Lagrangian for the constrained normalized residual. This extends the analysis of the eigenproblem and leads to derivation of more efficient algorithms compared to the original formulation. Additional algorithms based on bisection search and a standard L-curve approach are presented. These algorithms vary with respect to the parameters that need to be prescribed. Numerical and convergence results supporting the different versions and contrasting with RTLSQEP are presented.",
keywords = "Ill-posedness, Rayleigh quotient iteration, Regularization, Total least squares",
author = "Rosemary Renaut and Hongbin Guo",
year = "2005",
doi = "10.1137/S0895479802419889",
language = "English (US)",
volume = "26",
pages = "457--476",
journal = "SIAM Journal on Matrix Analysis and Applications",
issn = "0895-4798",
publisher = "Society for Industrial and Applied Mathematics Publications",
number = "2",

}

TY - JOUR

T1 - Efficient algorithms for solution of regularized total least squares

AU - Renaut, Rosemary

AU - Guo, Hongbin

PY - 2005

Y1 - 2005

N2 - Error-contaminated systems Ax ≈ b, for which A is ill-conditioned, are considered. Such systems may be solved using Tikhonov-like regularized total least squares (RTLS) methods. Golub, Hansen, and O'Leary [SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185-194] presented a parameter-dependent direct algorithm for the solution of the augmented Lagrange formulation for the RTLS problem, and Sima, Van Huffel, and Golub [Regularized Total Least Squares Based on Quadratic Eigenvalue Problem Solvers, Tech. Report SCCM-03-03, SCCM, Stanford University, Stanford, CA, 2003] have introduced a technique for solution based on a quadratic eigenvalue problem, RTLSQEP. Guo and Renaut [A regularized total least squares algorithm, in Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002, pp. 57-66] derived an eigenproblem for the RTLS which can be solved using the iterative inverse power method. Here we present an alternative derivation of the eigenproblem for constrained TLS through the augmented Lagrangian for the constrained normalized residual. This extends the analysis of the eigenproblem and leads to derivation of more efficient algorithms compared to the original formulation. Additional algorithms based on bisection search and a standard L-curve approach are presented. These algorithms vary with respect to the parameters that need to be prescribed. Numerical and convergence results supporting the different versions and contrasting with RTLSQEP are presented.

AB - Error-contaminated systems Ax ≈ b, for which A is ill-conditioned, are considered. Such systems may be solved using Tikhonov-like regularized total least squares (RTLS) methods. Golub, Hansen, and O'Leary [SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185-194] presented a parameter-dependent direct algorithm for the solution of the augmented Lagrange formulation for the RTLS problem, and Sima, Van Huffel, and Golub [Regularized Total Least Squares Based on Quadratic Eigenvalue Problem Solvers, Tech. Report SCCM-03-03, SCCM, Stanford University, Stanford, CA, 2003] have introduced a technique for solution based on a quadratic eigenvalue problem, RTLSQEP. Guo and Renaut [A regularized total least squares algorithm, in Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002, pp. 57-66] derived an eigenproblem for the RTLS which can be solved using the iterative inverse power method. Here we present an alternative derivation of the eigenproblem for constrained TLS through the augmented Lagrangian for the constrained normalized residual. This extends the analysis of the eigenproblem and leads to derivation of more efficient algorithms compared to the original formulation. Additional algorithms based on bisection search and a standard L-curve approach are presented. These algorithms vary with respect to the parameters that need to be prescribed. Numerical and convergence results supporting the different versions and contrasting with RTLSQEP are presented.

KW - Ill-posedness

KW - Rayleigh quotient iteration

KW - Regularization

KW - Total least squares

UR - http://www.scopus.com/inward/record.url?scp=17444432191&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=17444432191&partnerID=8YFLogxK

U2 - 10.1137/S0895479802419889

DO - 10.1137/S0895479802419889

M3 - Article

AN - SCOPUS:17444432191

VL - 26

SP - 457

EP - 476

JO - SIAM Journal on Matrix Analysis and Applications

JF - SIAM Journal on Matrix Analysis and Applications

SN - 0895-4798

IS - 2

ER -