Microsoft Technology Licensing, LLC

Ioan Bogdan Crivat

C James MacLennan

Raman S Iyer

Cristian Petculescu

Amir Netz

Marius Dumitru

Zhaohui Tang

Paul Jonathon Sanders

Irina Gorbach

Pyungchul Kim

Richard R Tkachuk

Akshai Mirchandani

Thulusalamatom Krishnamurthi Anand

Rong Jian Guan

Mosha Pasumansky

Alexander Berger

Thomas P Conlon

Scott Conrad Oveson

Adrian Dumitrascu

Andriy Garbuzov

Elena Daniela Cristofor

Chu Xu

Valeri Kim

Sergei Gringauze

The subject disclosure relates to column based data encoding where raw data to be compressed is organized by columns, and then, as first and second layers of reduction of the data size, dictionary encoding and/or value encoding are applied to the data as organized by columns, to create integer sequences that correspond to the columns. Next, a hybrid greedy run length encoding and bit packing compression algorithm further compacts the data according to an analysis of bit savings. Synergy of the hybrid data reduction techniques in concert with the column-based organization, coupled with gains in scanning and querying efficiency owing to the representation of the compact data, results in substantially improved data compression at a fraction of the cost of conventional systems.

Efficient column based data encoding for large-scale data storage

Systems and methodologies for identification of factors that cause significant shifts in transactions in a relational store and/or OLAP environment. Transactions are grouped into significant categories defined across the whole data space, to detect interesting sub spaces transactions. Subsequently, sub spaces that show strong variance between two slices can be selected, followed by grouping the subspaces in sub reports to measure the coverage for each sub report. A final report can then be generated that contains list of sub-reports detected in the previous acts.

Explaining changes in measures thru data mining

Systems and methods that supply extensibility mechanisms for analysis services, via a plug-in component that enables additional functionalities. The plug-in component provide additional custom logic for the analysis services unified dimensional model (UDM). Accordingly, server functionalities can be extended in an agile manner, and without a requirement for a new release, for example.

Extensibility mechanism for analysis services unified dimensional model

Data expressed as tabular data having columns and rows can be analyzed and data determined to be an exception can be flagged. In addition, reasons for flagging such data as exceptions can be presented to a user to facilitate further analysis and action on the data. A predictive analysis component can utilize a clustering algorithm with predictive capabilities to autonomously analyze the data. Periodic re-analysis of the data can be performed to determine if exceptions have changed based on new or modified data.

Detecting and displaying exceptions in tabular data

Fields contained in data expressed as tabular data having columns and rows can initially be marked as exceptions, wherein a column within a row can be the potential cause of the exception. A user configurable parameter can be utilized to change the sensitivity or allowable exceptions for each row and/or column, to increase or decrease the number of exceptions detected. As data within each field are modified, added or deleted, or when the configurable parameter is changed, the exceptions marked can be automatically updated. Such updated exceptions can be the same or different from the initially marked exceptions. As such, a user can evaluate data and determine whether various changes within the data will change various outcomes.

Dynamically detecting exceptions based on data changes

Seeking goals in data that can be expressed as rows and columns is provided through predictive analytics. If a desired goal is achievable, the changes to the rows and/or columns that can achieve the goal are presented to a user. If the desired goal is not achievable, an error message or other indicator can be presented to the user. Predictive analytics can include a predictive algorithm, various data mining techniques, or other predictive techniques. A confidence metric of a goal-seek result can be normalized to estimate the degree of confidence that a particular change will yield the desired outcome.

Goal seeking using predictive analytics

A system that effectuates fetching a complete set of relational data into a mining services server and subsequently defining desired partitions upon the fetched data is provided. In accordance with the innovation, the data can be locally cached and partitioned therefrom. Accordingly, upon the same mining structure (e.g., cache) that has been partitioned, the novel innovation can build mining models for each partition. In other words, the innovation can employ the concept of mining structure as a data cache while manipulating only partitions of this cache in certain operations. The innovation can be employed in scenarios where a user wants to train a mining model using only data points that satisfy a particular Boolean condition, a user wants to split the training set into multiple partitions (e.g., training/testing) and/or a user wants to perform a data mining procedure known as 'N-fold cross validation.'

Partitioning of data mining training set

The subject invention relates to systems and methods that extend the network data access capabilities of mark-up language protocols. In one aspect, a network data transfer system is provided. The system includes a protocol component that employs a computerized mark-up language to facilitate data interactions between network components, whereby the data interactions were previously limited or based on a statement command associated with the markup language. An extension component operates with the protocol component to support the data transactions, where the extension component supplies at least one other command from the statement command to facilitate the data interactions.

Systems and methods of utilizing and expanding standard protocol

Architecture for analyzing pattern shifts in data patterns of data mining models and outputting the results. This allows comparing and describing differences between two semantically similar sets of patterns (or mining models), and for analyzing historical changes in versions of the same model or differences in patterns found by two or more different algorithms applied to the same data. The architecture can also facilitate explaining data patterns that shift over time and over different data populations, and between versions of the same model that use different algorithms. A model component is employed for storing data mining models have respective sets of data patterns obtained from a dataset, and an analysis component analyzes the sets of the data patterns for difference data therebetween. The dataset can be a subsample of a larger set of data and can be analyzed by the analysis component over a time period.

Analyzing mining pattern evolutions by comparing labels, algorithms, or data patterns chosen by a reasoning component

A standard mechanism for directly accessing unstructured data types (e.g., image, audio, video, gene sequencing and text data) in accordance with data mining operations is provided. The subject innovation can enable access to unstructured data directly from within the data mining engine or tool. Accordingly, the innovation enables multiple vendors to provide algorithms for mining unstructured data on a data mining platform (e.g., an SQL-brand server), thereby increasing adoption. As well, the subject innovation allows users to directly mine unstructured data that is not fixed-length, without pre-processing and tokenizing the data external to the data mining engine. In accordance therewith, the innovation can provide a mechanism to expand declarative language content types to include an 'unstructured' data type thereby enabling a user and/or application to affirmatively designate mining data as an unstructured type.

Unstructured data in a mining model language

Architecture that facilitates syntax processing for data mining statements. The system includes a syntax engine that receives as an input a query statement which, for example, is a data mining request. The statement can be generated from many different sources, e.g., a client application and/or a server application, and requests query processing of a data source (e.g., a relational database) to return a result set. The syntax engine includes a binding component that converts the query statement into an encapsulated statement in accordance with a predefined grammar. The encapsulated statement includes both data and data operations to be performed on the data of the data source, and which is understood by the data source. An execution component processes the encapsulated statement against the data source to return the desired result set.

Using a rowset as a query parameter

A system that facilitates data mining comprises a reception component that receives command(s) in a declarative language that relate to utilizing an output of a first data mining model as an input to a second data mining model. An implementation component analyzes the received command(s) and implements the command(s) with respect to the first and second data mining models. In another aspect of the subject invention, the reception component can receive further command(s) in a declarative language with respect to causing one or more of the first and second data mining models to output a prediction, the prediction desirably generated without prediction input, the implementation component causes the one or more of the first and second data mining models to output the prediction.

Systems and methods that facilitate data mining

The subject disclosure pertains to extensible data mining systems, means, and methodologies. For example, a data mining system is disclosed that supports plug-in or integration of non-native mining algorithms, perhaps provided by third parties, such that they function the same as built-in algorithms. Furthermore, non-native data mining viewers may also be seamlessly integrated into the system for displaying the results of one or more algorithms including those provided by third parties as well as those built-in. Still further yet, support is provided for extending data mining languages to include user-defined functions (UDFs).

Growing community of inventors

Redmond, WA, United States of America

Ioan Bogdan Crivat