Microsoft Technology Licensing, LLC

Cory Alan Reina

Paul S Bradley

Usama M Fayyad

Raman Narayanan

Girish Bablani

One exemplary embodiment of a scalable clustering algorithm accesses a database of records having attributes or data fields of both enumerated discrete and ordered values and brings a portion of the data records into a rapid access memory. A cluster model for the data includes a table of probabilities for the enumerated, discrete data fields of the data records. The cluster model for data fields that are ordered comprises a mean and spread of the cluster. The cluster model is updated from the database records brought into the rapid access memory. At least some of the database records in the rapid access memory are summarized and stored within the rapid access memory. A criteria is then evaluated to determine if further data should be accessed from the database to further cluster data records in the database. Based on the evaluating step, additional database records in the database are accessed and brought into the rapid access memory for further updating of the cluster model.

Scalable system for clustering of large databases having mixed data attributes

A data mining system for use in finding clusters of data items in a database or any other data storage medium. The clusters are used in categorizing the data in the database into K different clusters within each of M models. An initial set of estimates (or guesses) of the parameters of each model to be explored (e.g. centriods in K-means), of each cluster are provided from some source. Then a portion of the data in the database is read from a storage medium and brought into a rapid access memory buffer whose size is determined by the user or operating system depending on available memory resources. Data contained in the data buffer is used to update the original guesses at the parameters of the model in each of the K clusters over all M models. Some of the data belonging to a cluster is summarized or compressed and stored as a reduced form of the data representing sufficient statistics of the data. More data is accessed from the database and the models are updated. An updated set of parameters for the clusters is determined from the summarized data (sufficient statistics) and the newly acquired data. Stopping criteria are evaluated to determine if further data should be accessed from the database. If further data is needed to characterize the clusters, more data is gathered from the database and used in combination with already compressed data until the stopping criteria has been met.

Scalable system for clustering of large databases

In one exemplary embodiment the invention provides a data mining system for use in finding clusters of data items in a database or any other data storage medium. Before the data evaluation begins a choice is made of the number M of models to be explored, and the number of clusters (K) of clusters within each of the M models. The clusters are used in categorizing the data in the database into K different clusters within each model. An initial set of estimates for a data distribution of each model to be explored is provided. Then a portion of the data in the database is read from a storage medium and brought into a rapid access memory buffer whose size is determined by the user or operating system depending on available memory resources. Data contained in the data buffer is used to update the original model data distributions in each of the K clusters over all M models. Some of the data belonging to a cluster is summarized or compressed and stored as a reduced form of the data representing sufficient statistics of the data. More data is accessed from the database and the models are updated. An updated set of parameters for the clusters is determined from the summarized data (sufficient statistics) and the newly acquired data. Stopping criteria are evaluated to determine if further data should be accessed from the database.

Scalable system for expectation maximization clustering of large databases

In one exemplary embodiment the invention provides a data mining system for use in evaluating data in a database. Before the data evaulation begins a choice is made of a cluster number K for use in categorizing the data in the database into K different clusters and initial guesses at the means, or centriods, of each cluster are provided. Then a portion of the data in the database is read from a storage medium and brought into a rapid access memory. Data contained in the data portion is used to update the original guesses at the centroids of each of the K clusters. Some of the data belonging to a cluster is summarized or compressed and stored as a summarization of the data. More data is accessed from the database and assigned to a cluster. An updated mean for the clusters is determined from the summarized data and the newly acquired data. A stopping criteria is evaluated to determine if further data should be accessed from the database. If further data is needed to characterize the clusters, more data is gathered from the database and used in combination with already compressed data until the stopping criteria has been met.

Scalable system for K-means clustering of large databases

A method and apparatus for creating a base class and derived classes in an object-oriented computer programming environment that allows an application program to manipulate external data connections in a similar manner. The base class includes sufficient functionality for all external data connections regardless of the protocol of the data transfer mechanism that is utilized for the external data connection such that the specific function calls required from the application to implement each data transfer mechanism is hidden from the application. Thus, the application sees all external data connections as being similar because the application issues function calls only to the base class.

Methods and apparatus for creating a base class for manipulating

A system for sharing an interface between objects is described herein that saves significant mounts of system resources, such as memory, in an object oriented system. The interface shared by the system described herein is an interface containing methods that perform a drag and drop operation. In addition, since one instance of the interface is shared among multiple objects, the system centralizes the processing performed by the methods of the interface. This centralization includes centralizing storage and state information associated with the processing of the methods of the interface, thereby further reducing utilization of system resources. The centralized processing occurs in a system where operations are performed on objects in a distributed manner (i.e., on an object-by-object basis). In other words, in a system where operations are typically performed on an object-by-object basis, the system described herein saves system resources by providing a mechanism for centralizing the processing of operations that are to be performed on multiple objects.

Growing community of inventors

Kirkland, WA, United States of America

Cory Alan Reina