TY - GEN
T1 - FROS
T2 - 2021 IEEE International Symposium on Information Theory, ISIT 2021
AU - Yang, Yingzhen
AU - Li, Ping
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/7/12
Y1 - 2021/7/12
N2 - Randomized algorithms are important for solving large-scale optimization problems. In this paper, we propose Fast Regularized Optimization by Sketching (FROS) as an efficient solver for a general class of regularized optimization problems. FROS first generates a sketch of the original data matrix, then solves the sketched problem. Different from existing randomized algorithms, FROS handles general Frechet subdifferentiable regularization functions in an unified framework. It is proved that FROS achieves relative-error bounds for the approximation error between the optimization results of the sketched problem and that of the original problem for all convex and certain non-convex regularization. We further propose Iterative FROS which reduces the approximation error exponentially by iteratively invoking FROS. To our best knowledge, our results are among the few in approximation error of sketching algorithms for a broad class of optimization problems with general regularization. Experimental results demonstrate the effectiveness of the proposed FROS and Iterative FROS algorithms.
AB - Randomized algorithms are important for solving large-scale optimization problems. In this paper, we propose Fast Regularized Optimization by Sketching (FROS) as an efficient solver for a general class of regularized optimization problems. FROS first generates a sketch of the original data matrix, then solves the sketched problem. Different from existing randomized algorithms, FROS handles general Frechet subdifferentiable regularization functions in an unified framework. It is proved that FROS achieves relative-error bounds for the approximation error between the optimization results of the sketched problem and that of the original problem for all convex and certain non-convex regularization. We further propose Iterative FROS which reduces the approximation error exponentially by iteratively invoking FROS. To our best knowledge, our results are among the few in approximation error of sketching algorithms for a broad class of optimization problems with general regularization. Experimental results demonstrate the effectiveness of the proposed FROS and Iterative FROS algorithms.
UR - http://www.scopus.com/inward/record.url?scp=85115087618&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85115087618&partnerID=8YFLogxK
U2 - 10.1109/ISIT45174.2021.9517998
DO - 10.1109/ISIT45174.2021.9517998
M3 - Conference contribution
AN - SCOPUS:85115087618
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 2780
EP - 2785
BT - 2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 12 July 2021 through 20 July 2021
ER -