Sas Institute Inc.

Oliver Schabenberger

James Howard Goodnight

Scott Russell Pope

Georges H Guirguis

Steve Krueger

Xunlei Wu

Christopher Daniel Bailey

Christopher D Bailey

A data processing system having multiple executable threads is configured to generate a cross-product matrix in a single pass through data. An example system comprises memory for receiving the data, a processor having a plurality of executable threads, and software code for generating a cross-product matrix in a single pass through the data. The software code includes threaded variable levelization code for generating thread specific binary trees for classification variables, variable tree merge code for combining the thread-specific trees into overall trees for the classification variables, effect levelization code for generating sub-matrices of the cross-product matrix using the overall trees for the classification variables, and cross-product matrix generation code for generating the cross-product matrix by storing and ordering the elements of the sub-matrices in contiguous memory space.

Systems and methods for generating a cross-product matrix in a single pass through data using single pass levelization

Systems and methods are provided for generating multiple system state projections using a grid computing environment. A central coordinator software component executes on a root data processor and provides commands and data to a plurality of node coordinator software components. A node coordinator software component manages threads which execute on its associated node data processor and which perform a set of matrix operations. Stochastic simulations use results of the matrix operations to generate multiple state projections. Additional processing can be performed by the grid computing environment based upon the generated state projections, such as to develop possible change information for users.

Distributed systems and methods for state generation based on multi-dimensional data

This disclosure describes methods, systems, computer-readable media, and apparatuses for efficiently calculating group-by statistics. A data set that includes multiple entries is accessed. The multiple entries are grouped into group-by subsets which are formed on two or more group-by variables and which are subsets are subsets of the data set. Cardinality data is determined for each of the group-by subsets, wherein cardinality data represents a number of entries in a group-by subset. At least one summary of data in each of the group-by subsets is generated, wherein each of the summaries includes the cardinality data determined for the group-by subset. Objects for the group-by subsets are initialized such that the objects store the summaries. The objects may then be used to generate multiple statistical summaries of the data set.

Methods and systems to operate on group-by sets with high cardinality

In accordance with the teachings described herein, systems and methods are provided for estimating quantiles for data stored in a distributed system. In one embodiment, an instruction is received to estimate a specified quantile for a variate in a set of data stored at a plurality of nodes in the distributed system. A plurality of data bins for the variate are defined that are each associated with a different range of data values in the set of data. Lower and upper quantile bounds for each of the plurality of data bins are determined based on the total number of data values that fall within each of the plurality of data bins. The specified quantile is estimated based on an identified one of the plurality of data bins that includes the specified quantile based on the lower and upper quantile bounds.

Systems and methods for quantile determination in a distributed data system

Systems and methods for quantile estimation in a distributed data system

Systems and methods are provided for a data processing system having multiple executable threads that is configured to generate a cross-product matrix in a single pass through data to be analyzed. An example system comprises memory for receiving the data to be analyzed, a processor having a plurality of executable threads for executing code to analyze data, and software code for generating a cross-product matrix in a single pass through data to be analyzed. The software code includes threaded variable levelization code for generating a plurality of thread specific binary trees for a plurality of classification variables, variable tree merge code for combining a plurality of the thread-specific trees into a plurality of overall trees for the plurality of classification variables, effect levelization code for generating sub-matrices using the plurality of the overall trees for the plurality of classification variables, and cross-product matrix generation code for generating the cross-product matrix.

Systems and methods are provided for a grid computing system that performs analytical calculations on data stored in a distributed database system. A grid-enabled software component at a control node is configured to invoke database management software (DBMS) at the control node to cause the DBMS at a plurality of the worker nodes to make available data to the grid-enabled software component local to its node; instruct the grid-enabled software components at the plurality of worker nodes to perform an analytical calculation on the received data and to send the results of the data analysis to the grid-enabled software component at the control node; and assemble the results of the data analysis performed by the grid-enabled software components at the plurality of worker nodes.

Growing community of inventors

Cary, NC, United States of America

Oliver Schabenberger