TY - JOUR
T1 - Efficient sparse generalized multiple kernel learning
AU - Yang, Haiqin
AU - Xu, Zenglin
AU - Ye, Jieping
AU - King, Irwin
AU - Lyu, Michael R.
N1 - Funding Information:
Manuscript received December 3, 2009; revised November 22, 2010; accepted December 23, 2010. Date of publication January 20, 2011; date of current version March 2, 2011. This work was substantially supported by two grants from the Research Grants Council of the Hong Kong SAR, China (Project CUHK4128/08E and Project CUHK4154/10E), and by funding from Google Focused Grant Project “Mobile 2014.” H. Yang, I. King, and M. R. Lyu are with the Department of Computer Science and Engineering, Chinese University of Hong Kong, Hong Kong (e-mail: hqyang@cse.cuhk.edu.hk; king@cse.cuhk.edu.hk; lyu@cse.cuhk.edu.hk).
PY - 2011/3
Y1 - 2011/3
N2 - Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L1-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the Lp-norm (p < 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L1-norm and the squared L2-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L1-norm or the L2-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.
AB - Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L1-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the Lp-norm (p < 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L1-norm and the squared L2-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L1-norm or the L2-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.
KW - Grouping effect
KW - Kernel methods
KW - level method
KW - multiple Kernel learning
UR - http://www.scopus.com/inward/record.url?scp=79952183228&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79952183228&partnerID=8YFLogxK
U2 - 10.1109/TNN.2010.2103571
DO - 10.1109/TNN.2010.2103571
M3 - Article
C2 - 21257374
AN - SCOPUS:79952183228
SN - 1045-9227
VL - 22
SP - 433
EP - 446
JO - IEEE Transactions on Neural Networks
JF - IEEE Transactions on Neural Networks
IS - 3
M1 - 5696759
ER -