Microsoft Technology Licensing, LLC

C James MacLennan

Ioan Bogdan Crivat

Raman S Iyer

Zhaohui Tang

Pyungchul Kim

Marius Dumitru

Cristian Petculescu

Paul Jonathon Sanders

Irina Gorbach

Jeffrey Bernhardt

Akshai Mirchandani

Richard R Tkachuk

Thulusalamatom Krishnamurthi Anand

Rong Jian Guan

Christopher A Meek

David Maxwell Chickering

Mosha Pasumansky

Alexander Berger

Jesper B Lind

Thomas P Conlon

Scott Conrad Oveson

Adrian Dumitrascu

Donald Munro Farmer

Andriy Garbuzov

Andy Linfoot

Elena Daniela Cristofor

Shuvro Mitra

Chu Xu

Valeri Kim

Sergei Gringauze

An intermediate representation of a workflow that comprises software functions may be generated to efficiently perform data marshalling. The workflow is analyzed, including identifying that a first software function is implemented in a first language, a second software function is implemented in a second language, and a third software function is not explicitly implemented in an implementation language. Factors associated with the software functions are analyzed, including implementation languages of the software functions. Based on the analysis of the factors, an implementation language is assigned to the third software function that comprises either the first or second language. Based on the analysis of the workflow, an intermediate representation of the workflow is generated that represents each of the plurality of software functions using declarative language. Generating the intermediate representation includes configuring the third software function to be implemented in the assigned implementation language upon compiling the intermediate representation.

Data marshalling optimization via intermediate representation of workflows

Described is time-weighted blending of the results of time series algorithms in a manner that changes their relative weights based on the prediction time. The prediction values from each algorithm are mathematically blended into a forecast result corresponding to the desired time of prediction. In this manner, an ARTXP algorithm that provides accurate near term predictions is given more weight than an ARIMA for near term predictions, and less relative weight for long term predictions. An example exponential function to compute the relative weights is described; the function corresponds to a curve having a control variable for the slope and the start of the curve, and constant coefficients, with the weights based (in part) on the prediction time. A user-provided parameter may also affect the relative weights used in the blending result.

Forecasting by blending algorithms to optimize near term and long term predictions

Various technologies and techniques are disclosed for text based schema discovery and information extraction. Documents are analyzed to identify sections of the documents and a relationship between the sections. Statistics are stored regarding occurrences of items in the documents. A probabilistic model is generated based on the stored statistics. A database schema is generated with a plurality of tables based upon the probabilistic model. The documents are analyzed against the probabilistic model to determine how the documents map to the tables generated from the database schema. The tables are populated from the documents based on a result of the analysis against the probabilistic model.

Text based schema discovery and information extraction

Systems and methods that supply extensibility mechanisms for analysis services, via a plug-in component that enables additional functionalities. The plug-in component provide additional custom logic for the analysis services unified dimensional model (UDM). Accordingly, server functionalities can be extended in an agile manner, and without a requirement for a new release, for example.

Extensibility mechanism for analysis services unified dimensional model

Data expressed as tabular data having columns and rows can be analyzed and data determined to be an exception can be flagged. In addition, reasons for flagging such data as exceptions can be presented to a user to facilitate further analysis and action on the data. A predictive analysis component can utilize a clustering algorithm with predictive capabilities to autonomously analyze the data. Periodic re-analysis of the data can be performed to determine if exceptions have changed based on new or modified data.

Detecting and displaying exceptions in tabular data

Fields contained in data expressed as tabular data having columns and rows can initially be marked as exceptions, wherein a column within a row can be the potential cause of the exception. A user configurable parameter can be utilized to change the sensitivity or allowable exceptions for each row and/or column, to increase or decrease the number of exceptions detected. As data within each field are modified, added or deleted, or when the configurable parameter is changed, the exceptions marked can be automatically updated. Such updated exceptions can be the same or different from the initially marked exceptions. As such, a user can evaluate data and determine whether various changes within the data will change various outcomes.

Dynamically detecting exceptions based on data changes

Seeking goals in data that can be expressed as rows and columns is provided through predictive analytics. If a desired goal is achievable, the changes to the rows and/or columns that can achieve the goal are presented to a user. If the desired goal is not achievable, an error message or other indicator can be presented to the user. Predictive analytics can include a predictive algorithm, various data mining techniques, or other predictive techniques. A confidence metric of a goal-seek result can be normalized to estimate the degree of confidence that a particular change will yield the desired outcome.

Goal seeking using predictive analytics

A system that effectuates fetching a complete set of relational data into a mining services server and subsequently defining desired partitions upon the fetched data is provided. In accordance with the innovation, the data can be locally cached and partitioned therefrom. Accordingly, upon the same mining structure (e.g., cache) that has been partitioned, the novel innovation can build mining models for each partition. In other words, the innovation can employ the concept of mining structure as a data cache while manipulating only partitions of this cache in certain operations. The innovation can be employed in scenarios where a user wants to train a mining model using only data points that satisfy a particular Boolean condition, a user wants to split the training set into multiple partitions (e.g., training/testing) and/or a user wants to perform a data mining procedure known as 'N-fold cross validation.'

Partitioning of data mining training set

The subject invention relates to systems and methods to extend the capabilities of declarative data modeling languages. In one aspect, a declarative data modeling language system is provided. The system includes a data modeling language component that generates one or more data mining models to extract predictive information from local or remote databases. A language extension component facilitates modeling capability in the data modeling language by providing a data sequence model or a time series model within the data modeling language to support various data mining applications.

Modeling sequence and time series data in predictive analytics

The subject invention relates to systems and methods that extend the network data access capabilities of mark-up language protocols. In one aspect, a network data transfer system is provided. The system includes a protocol component that employs a computerized mark-up language to facilitate data interactions between network components, whereby the data interactions were previously limited or based on a statement command associated with the markup language. An extension component operates with the protocol component to support the data transactions, where the extension component supplies at least one other command from the statement command to facilitate the data interactions.

Systems and methods of utilizing and expanding standard protocol

Architecture for analyzing pattern shifts in data patterns of data mining models and outputting the results. This allows comparing and describing differences between two semantically similar sets of patterns (or mining models), and for analyzing historical changes in versions of the same model or differences in patterns found by two or more different algorithms applied to the same data. The architecture can also facilitate explaining data patterns that shift over time and over different data populations, and between versions of the same model that use different algorithms. A model component is employed for storing data mining models have respective sets of data patterns obtained from a dataset, and an analysis component analyzes the sets of the data patterns for difference data therebetween. The dataset can be a subsample of a larger set of data and can be analyzed by the analysis component over a time period.

Analyzing mining pattern evolutions by comparing labels, algorithms, or data patterns chosen by a reasoning component

A language schema that integrates multidimensional extensions (e.g., MDX) and data mining extensions (e.g., DMX) for performing data mining operations on data residing in OLAP cubes. The schema provides that the &#x3c;source-data-query&#x3e; can not only be a relational query, rather a multidimensional query formed using MDX, for example. The operations of model creation, training and prediction are described.

Combining multidimensional expressions and data mining extensions to mine OLAP cubes

A standard mechanism for directly accessing unstructured data types (e.g., image, audio, video, gene sequencing and text data) in accordance with data mining operations is provided. The subject innovation can enable access to unstructured data directly from within the data mining engine or tool. Accordingly, the innovation enables multiple vendors to provide algorithms for mining unstructured data on a data mining platform (e.g., an SQL-brand server), thereby increasing adoption. As well, the subject innovation allows users to directly mine unstructured data that is not fixed-length, without pre-processing and tokenizing the data external to the data mining engine. In accordance therewith, the innovation can provide a mechanism to expand declarative language content types to include an 'unstructured' data type thereby enabling a user and/or application to affirmatively designate mining data as an unstructured type.

Unstructured data in a mining model language

Systems and methods that cleanse data in Extract, Transform, Load environments (ETL), via employing an outlier detect component that is positioned in data pipeline to data warehouse(s). Such outlier detect component employs a cluster mining model to split data into normal and outlier data. Different predictive models can be employed to detect outliers in different data slices to enhance the accuracy of the predictions. In addition, a graphical user interface (GUI) enables a user to interact with cluster groups that are created and/or analyzed by the outlier detect component.

Transform for outlier detection in extract, transfer, load environment

The subject invention leverages scaleable itemsets and/or association rules to provide dynamic adjustment of memory usage. This allows the subject invention to provide association rules and/or itemsets with the highest support while utilizing a bounded amount of memory. Thus, a data analysis system and/or method utilizing the subject invention can self-adjust to provide the best association rules and/or itemsets based on available system resources. One instance of the subject invention employs dynamically adjustable minimum support values for data itemsets and/or association rules to facilitate in compensating for memory availability. In yet another instance of the subject invention a prefix tree data structure is utilized to facilitate in constructing itemsets. Memory utilization is then adjusted via pruning and/or reallocation of counter vectors and/or pointer vectors and/or reallocation of nodes of the prefix tree data structure for scaleable data itemsets and/or association rules.

Growing community of inventors

Redmond, WA, United States of America

C James MacLennan