Feature selection and hyperparameter optimization using lds

Abstract

A computer-implemented method for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning is provided including setting a first solution having first hyperparameters and a first set of features of a plurality of features of a training data set, initializing a weight table providing a score for each feature of the first set of features, and initializing a discrepancy. The method further includes performing a limited discrepancy search (LDS), according to an order based on the weight table, to obtain a second solution having second hyperparameters and a second set of features by swapping the first set of features and switching the first hyperparameters from the first solution with the discrepancy, while updating the weight table during LDS, comparing the second solution with the first solution, and obtaining a new solution with improved features and hyperparameters, as an optimized solution.