|Class KRR implements Kernel Ridge Regression - a regularized least square method for classification and regression. |
Class KRR implements Kernel Ridge Regression - a regularized least square method for classification and regression.
It is similar to support vector machines (cf. CSVM). However in contrast to SVMs a different objective is optimized that leads to a dense solution (thus not only a few support vectors are active in the end but all training examples). This makes it only applicable to rather few (a couple of thousand) training examples. In case a linear kernel is used RR is closely related to Fishers Linear Discriminant (cf. LDA).
Internally (for linear kernels) it is solved via minimizing the following system
which is boils down to solving a linear system
and in the kernel case
where K is the kernel matrix and y the vector of labels. The expressed solution can again be written as a linear combination of kernels (cf. CKernelMachine) with bias .