A note on the numerical solutions of kernel-based learning problems

M. Scandella, M. Mazzoleni, S. Formentin, F. Previdi

In the last decade, kernel-based learning approaches
typically employed for classification and regression have shown
outstanding performance also in dynamic system identification.
The typical way to compute the solution of this learning problem
subsumes the inversion of the kernel matrix. However, due
to limited machine precision, this might not be possible in
many practical applications. In this work, we analyze the above
problem and show that the typical estimate is just one of the
possible infinite solutions that can be leveraged, considering both
the supervised and the semi-supervised settings. We show under
which conditions the infinite solutions are equivalent and, if it is
not the case, we provide a bound on the mismatch between two
generic solutions. Then, we propose two specific solutions that
are particularly suited to boost sparsity or performance.