Microsoft Technology Licensing, LLC

Edward George Essey

John Joseph Duffy

Andrey Shur

Charles David Callahan, Ii

Constantin Stanciu

Patrick H Halstead

Igor Ostrovsky

Stephen Harris Toub

Jonathan David Perlow

Hazim Shafi

Shiraz Jitendra Cupala

David S Callahan

Brian G O'Connor

Huseyin Serkan Yildiz

Mike Liddell

Kamaljit S Bath

Nathaniel Stott

Alexandru Croicu

William T Colburn

Pooja Nagpal

A query that identifies an input data source is rewritten to contain data parallel operations that include partitioning and merging. The input data source is partitioned into a plurality of initial partitions. A parallel repartitioning operation is performed on the initial partitions to generate a plurality of secondary partitions. A parallel execution of the query is performed using the secondary partitions to generate a plurality of output sets. The plurality of output sets are merged into a merged output set.

Partitioning and repartitioning for data parallel operations

The present invention extends to methods, systems, and computer program products for indicating parallel operations with user-visible events. Event markers can be used to indicate an abstracted outer layer of execution as well as expose internal specifics of parallel processing systems, including systems that provide data parallelism. Event markers can be used to show a variety of execution characteristics including higher-level markers to indicate the beginning and end of an execution program (e.g., a query). Inside the execution program (query) individual fork/join operations can be indicated with sub-levels of markers to expose their operations. Additional decisions made by an execution engine, such as, for example, when elements initially yield, when queries overlap or nest, when the query is cancelled, when the query bails to sequential operation, when premature merging or re-partitioning are needed can also be exposed.

Indicating parallel operations with user-visible events

Determining correspondence, association, and/or correlation between nodes of electronic forms based on schemas of those forms is described. Importing, merging, and/or annotating data from one electronic form into another electronic form based on correspondence, association, and/or correlation between nodes is also described.

Correlation, association, or correspondence of electronic forms

A query that includes a search operator and that identifies an input data source is received. The input data source is partitioned into a plurality of partitions. A parallel search through the partitions is performed for an element that could halt the search. The parallel search is performed using a plurality of parallel workers. One of the parallel workers generates a notification when the element is found by that worker. The notification notifies the other parallel workers that the search could be halted. Each of the parallel workers generates an output set based on results of the search. The output sets are merged into a merged output set.

Data parallel searching

A system and method that enables promotion of data from within a structured data file to a library is described. This system and method can also enable demotion of data from a library to within a structured data file. Further, this system and method can copy data from a structured data file, perform a function on the copied data, and promote a result of the performed function to a library.

Promoting data from structured data files

A method of analyzing a data parallel query includes receiving a user-specified data parallel query that includes a plurality of query operators. An operator type for each of the query operators is identified based on a type of parallel input data structure the operator operates on and a type of parallel output data structure the operator outputs. It is determined whether the query is a prohibited query based on the identified operator types.

Data parallel query analysis

A parallel execution of a query is performed with a plurality of producers using a partitioned data source, thereby producing output data. The output data is consumed with at least one consumer. The at least one consumer is monitored. A determination is made whether to stop producing output data by at least one of the producers prior to completion of the execution of the query based on the monitoring.

Data parallel production and consumption

Various technologies and techniques are disclosed for preserving input element ordering in data parallel operations. This ordering may be based on element ordinal position in the input or a programmer-specified key-selection routine that generates sortable keys for each input element. Complex data parallel operations are re-written to contain individual data parallel operations that introduce partitioning and merging. Each partition is then processed independently in parallel. The system ensures that downstream operations remember ordering information established by certain other operations, using techniques that vary depending upon which categories the consumer operations are in. Data is merged back into one output stream using a final merge process that is aware of the ordering established among data elements.

Order preservation in data parallel operations

A system and method that aggregates information from one electronic form into another electronic form is described. For sufficiently similar electronic forms, the system and method aggregates information from one electronic form into another automatically and without use of a transformation file. For electronic forms governed by substantially dissimilar schemas, the system and method automatically aggregates information from one electronic form into another with the aid of a transformation file.

Growing community of inventors

Seattle, WA, United States of America

Edward George Essey