TY - JOUR
T1 - Regularization parameter estimation for large-scale Tikhonov regularization using a priori information
AU - Renaut, Rosemary
AU - Hnětynková, Iveta
AU - Mead, Jodi
N1 - Funding Information:
The first author completed portions of this work while visiting the Institute of Computer Science at the Academy of Sciences of the Czech Republic and the Department of Informatics and Mathematical Modeling at the Technical University of Denmark. This work is supported by grants to the authors, for the first author NSF grants DMS 0513214 and DMS 0652833, the second author by research project MSM0021620839 financed by MSMT, the Charles University in Prague and for the third author by NSF grant EPS 0447689. We also thank the two referees whose comments helped with improving the clarity of the paper, and particularly with respect to explaining the difference between the presented and standard hybrid LSQR methods.
Copyright:
Copyright 2017 Elsevier B.V., All rights reserved.
PY - 2010/12/1
Y1 - 2010/12/1
N2 - Solutions of numerically ill-posed least squares problems Ax ≈ b for A ∈ Rmxn by Tikhonov regularization are considered. For D ∈ Rpxn, the Tikhonov regularized least squares functional is given by J(σ) = \\Ax - b||2 w + 1/σ2 ||D(x - x0) \\2 2where matrix W is a weighting matrix and X0 is given. Given a priori estimates on the covariance structure of errors in the measurement data b, the weighting matrix may be taken as W = Wb which is the inverse covariance matrix of the mean 0 normally distributed measurement errors e in b. If in addition X0 is an estimate of the mean value of x, and σ is a suitable statistically-chosen value, J evaluated at its minimizer x(σ) approximately follows a x 2 distribution with m̃ = m + p-n degrees of freedom. Using the generalized singular value decomposition of the matrix pair [Wb 1/2AD], σ can then be found such that the resulting J follows this x2 distribution. But the use of an algorithm which explicitly relies on the direct solution of the problem obtained using the generalized singular value decomposition is not practical for large-scale problems. Instead an approach using the Golub-Kahan iterative bidiagonalization of the regularized problem is presented. The original algorithm is extended for cases in which X0 is not available, but instead a set of measurement data provides an estimate of the mean value of b. The sensitivity of the Newton algorithm to the number of steps used in the Golub-Kahan iterative bidiagonalization, and the relation between the size of the projected subproblem and a are discussed. Experiments presented contrast the efficiency and robustness with other standard methods for finding the regularization parameter for a set of test problems and for the restoration of a relatively large real seismic signal. An application for image deblurring also validates the approach for large-scale problems. It is concluded that the presented approach is robust for both small and large-scale discretely ill-posed least squares problems.
AB - Solutions of numerically ill-posed least squares problems Ax ≈ b for A ∈ Rmxn by Tikhonov regularization are considered. For D ∈ Rpxn, the Tikhonov regularized least squares functional is given by J(σ) = \\Ax - b||2 w + 1/σ2 ||D(x - x0) \\2 2where matrix W is a weighting matrix and X0 is given. Given a priori estimates on the covariance structure of errors in the measurement data b, the weighting matrix may be taken as W = Wb which is the inverse covariance matrix of the mean 0 normally distributed measurement errors e in b. If in addition X0 is an estimate of the mean value of x, and σ is a suitable statistically-chosen value, J evaluated at its minimizer x(σ) approximately follows a x 2 distribution with m̃ = m + p-n degrees of freedom. Using the generalized singular value decomposition of the matrix pair [Wb 1/2AD], σ can then be found such that the resulting J follows this x2 distribution. But the use of an algorithm which explicitly relies on the direct solution of the problem obtained using the generalized singular value decomposition is not practical for large-scale problems. Instead an approach using the Golub-Kahan iterative bidiagonalization of the regularized problem is presented. The original algorithm is extended for cases in which X0 is not available, but instead a set of measurement data provides an estimate of the mean value of b. The sensitivity of the Newton algorithm to the number of steps used in the Golub-Kahan iterative bidiagonalization, and the relation between the size of the projected subproblem and a are discussed. Experiments presented contrast the efficiency and robustness with other standard methods for finding the regularization parameter for a set of test problems and for the restoration of a relatively large real seismic signal. An application for image deblurring also validates the approach for large-scale problems. It is concluded that the presented approach is robust for both small and large-scale discretely ill-posed least squares problems.
UR - http://www.scopus.com/inward/record.url?scp=78049324264&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=78049324264&partnerID=8YFLogxK
U2 - 10.1016/j.csda.2009.05.026
DO - 10.1016/j.csda.2009.05.026
M3 - Article
AN - SCOPUS:78049324264
SN - 0167-9473
VL - 54
SP - 3430
EP - 3445
JO - Computational Statistics and Data Analysis
JF - Computational Statistics and Data Analysis
IS - 12
ER -