TY - JOUR
T1 - Efficient estimation of regularization parameters via downsampling and the singular value expansion
T2 - Downsampling regularization parameter estimation
AU - Renaut, Rosemary
AU - Horst, Michael
AU - Wang, Yang
AU - Cochran, Douglas
AU - Hansen, Jakob
N1 - Publisher Copyright:
© 2016, Springer Science+Business Media Dordrecht.
PY - 2017/6/1
Y1 - 2017/6/1
N2 - The solution, x, of the linear system of equations Ax≈ b arising from the discretization of an ill-posed integral equation g(s)=∫H(s,t)f(t)dt with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution x(λ) approximating the Galerkin coefficients of f(t) is found as the minimizer of J(x)={‖Ax-b‖22+λ2‖Lx‖22}, where b is given by the Galerkin coefficients of g(s). x(λ) depends on the regularization parameter λ that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution x(λ) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.
AB - The solution, x, of the linear system of equations Ax≈ b arising from the discretization of an ill-posed integral equation g(s)=∫H(s,t)f(t)dt with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution x(λ) approximating the Galerkin coefficients of f(t) is found as the minimizer of J(x)={‖Ax-b‖22+λ2‖Lx‖22}, where b is given by the Galerkin coefficients of g(s). x(λ) depends on the regularization parameter λ that trades off between the data fidelity and the smoothing norm determined by L, here assumed to be diagonal and invertible. The Galerkin method provides the relationship between the singular value expansion of the continuous kernel and the singular value decomposition of the discrete system matrix for square integrable kernels. We prove that the kernel maintains square integrability under left and right multiplication by bounded functions and thus the relationship also extends to appropriately weighted kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution x(λ) independent of the sample size of the data. We prove that consistently down sampling both the system matrix and the data provides a small scale system that preserves the dominant terms of the right singular subspace of the system and can then be used to estimate the regularization parameter for the original system. When g(s) is directly measured via its Galerkin coefficients the regularization parameter is preserved across resolutions. For measurements of g(s) a scaling argument is required to move across resolutions of the systems when the regularization parameter is found using a regularization parameter estimation technique that depends on the knowledge of the variance in the data. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied to both the system of equations, and to the regularized system of equations.
KW - Ill-posed inverse problem
KW - Regularization parameter estimation
KW - Singular value decomposition
KW - Singular value expansion
KW - Tikhonov regularization
UR - http://www.scopus.com/inward/record.url?scp=84995755426&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84995755426&partnerID=8YFLogxK
U2 - 10.1007/s10543-016-0637-6
DO - 10.1007/s10543-016-0637-6
M3 - Article
AN - SCOPUS:84995755426
SN - 0006-3835
VL - 57
SP - 499
EP - 529
JO - BIT Numerical Mathematics
JF - BIT Numerical Mathematics
IS - 2
ER -