Efficient estimation of regularization parameters via downsampling and the singular value expansion: Downsampling regularization parameter estimation

Rosemary Renaut, Michael Horst, Yang Wang, Douglas Cochran, Jakob Hansen

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The solution, (Formula presented.), of the linear system of equations (Formula presented.) arising from the discretization of an ill-posed integral equation (Formula presented.) with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution (Formula presented.) approximating the Galerkin coefficients of f(t) is found as the minimizer of (Formula presented.), where (Formula presented.) is given by the Galerkin coefficients of g(s). (Formula presented.) depends on the regularization parameter (Formula presented.) that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution (Formula presented.) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.

Original languageEnglish (US)
Pages (from-to)1-31
Number of pages31
JournalBIT Numerical Mathematics
DOIs
StateAccepted/In press - Nov 19 2016

Fingerprint

Efficient Estimation
Regularization Parameter
Singular Values
Parameter estimation
Parameter Estimation
Integral equations
kernel
Galerkin methods
Singular value decomposition
Galerkin
Linear systems
Sampling
System of equations
Integral Equations
Coefficient
Discrepancy Principle
Generalized Cross-validation
Linear system of equations
Minimizer
Discrete Systems

Keywords

  • Ill-posed inverse problem
  • Regularization parameter estimation
  • Singular value decomposition
  • Singular value expansion
  • Tikhonov regularization

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications
  • Computational Mathematics
  • Applied Mathematics

Cite this

Efficient estimation of regularization parameters via downsampling and the singular value expansion : Downsampling regularization parameter estimation. / Renaut, Rosemary; Horst, Michael; Wang, Yang; Cochran, Douglas; Hansen, Jakob.

In: BIT Numerical Mathematics, 19.11.2016, p. 1-31.

Research output: Contribution to journalArticle

@article{38b9e0f3251d41049b4872879b34a275,
title = "Efficient estimation of regularization parameters via downsampling and the singular value expansion: Downsampling regularization parameter estimation",
abstract = "The solution, (Formula presented.), of the linear system of equations (Formula presented.) arising from the discretization of an ill-posed integral equation (Formula presented.) with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution (Formula presented.) approximating the Galerkin coefficients of f(t) is found as the minimizer of (Formula presented.), where (Formula presented.) is given by the Galerkin coefficients of g(s). (Formula presented.) depends on the regularization parameter (Formula presented.) that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution (Formula presented.) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.",
keywords = "Ill-posed inverse problem, Regularization parameter estimation, Singular value decomposition, Singular value expansion, Tikhonov regularization",
author = "Rosemary Renaut and Michael Horst and Yang Wang and Douglas Cochran and Jakob Hansen",
year = "2016",
month = "11",
day = "19",
doi = "10.1007/s10543-016-0637-6",
language = "English (US)",
pages = "1--31",
journal = "BIT Numerical Mathematics",
issn = "0006-3835",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Efficient estimation of regularization parameters via downsampling and the singular value expansion

T2 - Downsampling regularization parameter estimation

AU - Renaut, Rosemary

AU - Horst, Michael

AU - Wang, Yang

AU - Cochran, Douglas

AU - Hansen, Jakob

PY - 2016/11/19

Y1 - 2016/11/19

N2 - The solution, (Formula presented.), of the linear system of equations (Formula presented.) arising from the discretization of an ill-posed integral equation (Formula presented.) with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution (Formula presented.) approximating the Galerkin coefficients of f(t) is found as the minimizer of (Formula presented.), where (Formula presented.) is given by the Galerkin coefficients of g(s). (Formula presented.) depends on the regularization parameter (Formula presented.) that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution (Formula presented.) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.

AB - The solution, (Formula presented.), of the linear system of equations (Formula presented.) arising from the discretization of an ill-posed integral equation (Formula presented.) with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution (Formula presented.) approximating the Galerkin coefficients of f(t) is found as the minimizer of (Formula presented.), where (Formula presented.) is given by the Galerkin coefficients of g(s). (Formula presented.) depends on the regularization parameter (Formula presented.) that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution (Formula presented.) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.

KW - Ill-posed inverse problem

KW - Regularization parameter estimation

KW - Singular value decomposition

KW - Singular value expansion

KW - Tikhonov regularization

UR - http://www.scopus.com/inward/record.url?scp=84995755426&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84995755426&partnerID=8YFLogxK

U2 - 10.1007/s10543-016-0637-6

DO - 10.1007/s10543-016-0637-6

M3 - Article

AN - SCOPUS:84995755426

SP - 1

EP - 31

JO - BIT Numerical Mathematics

JF - BIT Numerical Mathematics

SN - 0006-3835

ER -