Unbiased predictive risk estimation of the Tikhonov regularization parameter: convergence with increasing rank approximations of the singular value decomposition

Rosemary Renaut, Anthony W. Helmstetter, Saeed Vatankhah

Research output: Contribution to journalArticle

Abstract

The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition.

Original languageEnglish (US)
JournalBIT Numerical Mathematics
DOIs
StatePublished - Jan 1 2019

Fingerprint

Tikhonov Regularization
Regularization Parameter
Singular value decomposition
Approximation
Term
Estimator
Generalized Cross-validation
Ill-posedness
Ill-posed Problem
Theoretical Analysis
Regularization
Efficient Algorithms
Sufficient
Lower bound
Verify
Converge
Dependent
Coefficient
Demonstrate
Simulation

Keywords

  • Inverse problems
  • Regularization parameter
  • Tikhonov regularization
  • Unbiased predictive risk estimation

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications
  • Computational Mathematics
  • Applied Mathematics

Cite this

@article{76cf3c1fa8c24cfdbafff33deaf6af80,
title = "Unbiased predictive risk estimation of the Tikhonov regularization parameter: convergence with increasing rank approximations of the singular value decomposition",
abstract = "The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition.",
keywords = "Inverse problems, Regularization parameter, Tikhonov regularization, Unbiased predictive risk estimation",
author = "Rosemary Renaut and Helmstetter, {Anthony W.} and Saeed Vatankhah",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s10543-019-00762-7",
language = "English (US)",
journal = "BIT Numerical Mathematics",
issn = "0006-3835",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Unbiased predictive risk estimation of the Tikhonov regularization parameter

T2 - convergence with increasing rank approximations of the singular value decomposition

AU - Renaut, Rosemary

AU - Helmstetter, Anthony W.

AU - Vatankhah, Saeed

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition.

AB - The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition.

KW - Inverse problems

KW - Regularization parameter

KW - Tikhonov regularization

KW - Unbiased predictive risk estimation

UR - http://www.scopus.com/inward/record.url?scp=85067945312&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067945312&partnerID=8YFLogxK

U2 - 10.1007/s10543-019-00762-7

DO - 10.1007/s10543-019-00762-7

M3 - Article

AN - SCOPUS:85067945312

JO - BIT Numerical Mathematics

JF - BIT Numerical Mathematics

SN - 0006-3835

ER -