Sas Institute Inc.

Georges H Guirguis

Scott Russell Pope

Oliver Schabenberger

Robert S Ray

Dominique Joseph Latour

Xiangqian Hu

Guixian Lin

Guy Blanc

A computing device creates a state of an analytic model. An analytic engine of an analytic model of an analytic model type is initialized. The analytic model is trained using a dataset and the analytic engine. A request to save a state of the analytic model is received. In response to receipt of the third indicator, the state of the trained analytic model is serialized. The state includes a reentry point name of a function of the analytic model type called to instantiate the trained analytic model. The serialized state is written to an output file. The written state is read from the output file. The state of the trained analytic model is restored using the read state. An analytic result is computed based on data in a second dataset different from the named dataset using the restored state of the trained analytic model. The computed analytic result is output.

Streamlined analytic model training and scoring system

In accordance with the teachings described herein, systems and methods are provided for estimating or determining quantiles for data stored in a distributed system. In one embodiment, an instruction is received to estimate or determine a specified quantile for a variate in a set of data stored at a plurality of nodes in the distributed system. A plurality of data bins for the variate are defined that are each associated with a different range of data values in the set of data. Lower and upper quantile bounds for each of the plurality of data bins are determined based on the total number of data values that fall within each of the plurality of data bins. The specified quantile is estimated or determined based on an identified one of the plurality of data bins that includes the specified quantile based on the lower and upper quantile bounds.

Systems and methods for quantile determination in a distributed data system using sampling

A computing device restores a state of an analytic model. A first indicator of a dataset is received. A second indicator is received of an analytic model state file in which a state of an analytic model is stored. A reentry point name of a function to restore the analytic model is extracted from the state stored in the analytic model state file indicated by the received second indicator. The function is called using the extracted reentry point name to restore the state of the analytic model. The dataset indicated by the received first indicator is scored using the restored state of the analytic model. Scoring results from the scored dataset are written to an output file.

Streamlined system to restore an analytic model state for training and scoring

In accordance with the teachings described herein, systems and methods are provided for estimating quantiles for data stored in a distributed system. In one embodiment, an instruction is received to estimate a specified quantile for a variate in a set of data stored at a plurality of nodes in the distributed system. A plurality of data bins for the variate are defined that are each associated with a different range of data values in the set of data. Lower and upper quantile bounds for each of the plurality of data bins are determined based on the total number of data values that fall within each of the plurality of data bins. The specified quantile is estimated based on an identified one of the plurality of data bins that includes the specified quantile based on the lower and upper quantile bounds.

Systems and methods for quantile determination in a distributed data system

Systems and methods for quantile estimation in a distributed data system

Systems and methods are provided for identifying data variable roles during initial data exploration. A variable type, unique data value count values, and an overflow count value are determined for a variable. The unique data value count values include a number of occurrences of each of a plurality of unique data values for the variable in a data set. The overflow count value is a number of occurrences of data values other than the plurality of unique data values for the variable in the data set. When a number of the plurality of unique data values is greater than a value for a high cardinality threshold, the variable is determined to be a high cardinality variable. When the variable is not determined to be the high cardinality variable, a class variable role is assigned to the variable. When the variable is determined to be the high cardinality variable, Whether or not the variable is a numeric variable type is determined based on the determined variable type. When the variable is determined to not be the numeric variable type, the overflow count value is compared to the unique data value count values to determine whether or not rare visible values occurred for the variable. When the determination is that rare visible values occurred for the variable, a record identifier variable role is assigned to the variable.

System and method for fast identification of variable roles during initial data exploration

Systems and methods are provided for identifying data variable rules during initial data exploration. In one example, a computer-implemented method of determining a role for a data variable is disclosed. The method comprises identifying to a plurality of data nodes a set of data records containing data values assigned to each data node, a maximum number of levels to record in a sorted data structure at the data nodes, and the data node responsible for each of a plurality of variables. The method further comprises receiving for each variable from the data node responsible for the variable a plurality of unique data values for the variable, a count for each of the unique data values and an overflow count for the variable, wherein the number of unique data values does not exceed the maximum number of levels. A role for a variable can be determined based upon the unique data values, counts and overflow count for the variable.

A computer-implemented system and method of non-linear modeling in a computer system having a limited precision processor is provided. A non-linear model is initialized by forming an objective function having one or more functional components and a marginal variance matrix. The model is then iteratively solved using the computer processor until it has converged to a feasible solution. In doing so, the feasibility of computing the objective function is evaluated by determining if the marginal variance matrix is positive definite, thereby indicating whether or not the computer processor is capable of calculating a feasible solution to the non-linear model. If the marginal variance matrix is positive definite, then the objective function and its gradient are computed using the marginal variance matrix. If the marginal variance matrix is not positive definite, then a surrogate marginal variance matrix is constructed that is positive definite and a surrogate objective function is constructed having components continuous first derivatives. The surrogate objective function and its gradient are then computed using the surrogate marginal variance matrix.

Growing community of inventors

Cary, NC, United States of America

Georges H Guirguis