Fast and accurate machine learning by applying efficient preconditioner to kernel ridge regression

Abstract

Accelerated machine learning using an efficient preconditioner for Kernel Ridge Regression (KRR). A plurality of anchor points may be selected by: projecting an initial kernel onto a random matrix in a lower dimensional space to generate a randomized decomposition of the initial kernel, permuting the randomized decomposition to reorder its columns and/or rows to approximate the initial kernel, and selecting anchor points representing a subset of the columns and/or rows based on their permuted order. A reduced-rank approximation kernel may be generated comprising the subset of columns and/or rows represented by the selected anchor points. A KRR system may be preconditioned using a preconditioner generated based on the reduced-rank approximation kernel. The preconditioned KRR system may be solved to train the machine learning model. This KRR technique may be executed without generating the KRR kernel, reducing processor and memory consumption.