Nec Laboratories America, Inc.

Vladimir Vapnik

Eric Cosatto

Hans Peter Graf

Akshay Vashist

Igor Durdanovic

Rauf Izmailov

Leon Bottou

Margaret A Miller, Legal Representative

Michael R Miller

A method and system for use in describing a phenomenon of interest. The method and system computes a decision rule for use in describing the phenomenon of interest using training data relating to the phenomenon of interest, labels for labeling the training data, and hidden information about the training data or directed distances obtained from the hidden information, as inputs.

System and method using hidden information

Disclosed is an improved technique for training a support vector machine using a distributed architecture. A training data set is divided into subsets, and the subsets are optimized in a first level of optimizations, with each optimization generating a support vector set. The support vector sets output from the first level optimizations are then combined and used as input to a second level of optimizations. This hierarchical processing continues for multiple levels, with the output of each prior level being fed into the next level of optimizations. In order to guarantee a global optimal solution, a final set of support vectors from a final level of optimization processing may be fed back into the first level of the optimization cascade so that the results may be processed along with each of the training data subsets. This feedback may continue in multiple iterations until the same final support vector set is generated during two sequential iterations through the cascade, thereby guaranteeing that the solution has converged to the global optimal solution. In various embodiments, various combinations of inputs may be used by the various optimizations. The individual optimizations may be processed in parallel.

Parallel support vector method and apparatus

A new machine learning technique is herein disclosed which generalizes the support vector machine framework. A separating hyperplane in a separating space is optimized in accordance with generalized constraints which dependent upon the clustering of the input vectors in the dataset.

Learning machine that considers global structure of data

A system and method for support vector machine plus (SVM+) computations include selecting a set of indexes for a target function to create a quadratic function depending on a number of variables, and reducing the number of variables to two in the quadratic function using linear constraints. An extreme point is computed for the quadratic function in closed form. A two-dimensional set is defined where the indexes determine whether a data point is in the two-dimensional set or not. A determination is made of whether the extreme point belongs to the two-dimensional set. If the extreme point belongs to the two-dimensional set, the extreme point defines a maximum and defines a new set of parameters for a next iteration. Otherwise, the quadratic function is restricted on at least one boundary of the two-dimensional set to create a one-dimensional quadratic function. The steps are repeated until the maximum is determined.

Generalized sequential minimal optimization for SVM+ computations

Disclosed is a parallel support vector machine technique for solving problems with a large set of training data where the kernel computation, as well as the kernel cache and the training data, are spread over a number of distributed machines or processors. A plurality of processing nodes are used to train a support vector machine based on a set of training data. Each of the processing nodes selects a local working set of training data based on data local to the processing node, for example a local subset of gradients. Each node transmits selected data related to the working set (e.g., gradients having a maximum value) and receives an identification of a global working set of training data. The processing node optimizes the global working set of training data and updates a portion of the gradients of the global working set of training data. The updating of a portion of the gradients may include generating a portion of a kernel matrix. These steps are repeated until a convergence condition is met. Each of the local processing nodes may store all, or only a portion of, the training data. While the steps of optimizing the global working set of training data, and updating a portion of the gradients of the global working set, are performed in each of the local processing nodes, the function of generating a global working set of training data is performed in a centralized fashion based on the selected data (e.g., gradients of the local working set) received from the individual processing nodes.

Growing community of inventors

Plainsboro, NJ, United States of America

Vladimir Vapnik