Microsoft Technology Licensing, LLC

José A Blakeley

Marc Todd Friedman

Alan D Halverson

Milind Madhukar Joshi

Haibo Lin

Nikhil Teletia

Raghunath Ramakrishnan

Zhicheng Yin

Jaliya Nishantha Ekanayake

Tao Guan

Yongchul Kwon

Jin Sun

Namyong Lee

Jose Aguilar Saborit

Ming Li

Aspects extend to methods, systems, and computer program products for optimally pipelining result sets with fault tolerance in distributed query execution. Distributed computing jobs are optimized by dividing the distributed computing jobs into one or more bubbles for execution. Each bubble can be independently executed, potentially in parallel with other bubbles, when resources to handle the bubble are available. Intra-bubble communication can be streamed between vertices within a bubble. Inter-bubble communication can be stored to durable storage. Bubbles provide a failure boundary for a job graph and re-executing a bubble along with storage of intermediate results in durable storage can be used to recover from failures. When a vertex inside a bubble fails, computation can resume by rescheduling the execution of the failed bubble from the durable inputs for that bubble. Durable storage provides a light-weight failover to handle non-deterministic behavior. Jobs can also leverage streaming to increase performance.

Optimizing pipelining result sets with fault tolerance in distributed query execution

The present invention extends to methods, systems, and computer program products for validations the configuration of distributed systems. Validation can include a collection of utilities that are systematically deployed across the nodes of an appliance. The utilities execute specified tasks, that when completed, validate the proper functioning state of hardware and/or software components of the appliance. The utilities generate logs, which can be aggregated to present a global view of the appliance.

Validating the configuration of distributed systems

The present invention extends to methods, systems, and computer program products for performing parallel joins on distributed database data. Embodiments of the invention include a phased semi-join reduction strategy using replication and shuffle operations to join a first and a second data source. A filter building phase uses replication and pushes down a 'Distinct' (e.g., SQL) operator to produce a list of join keys for the first data source (one side of the join). A shuffle phase for the second data source is modified to join to the key list produced in the first phase as a row filtering mechanism. A join phase then joins the first and second data sources.

Performing parallel joins on distributed database data

Growing community of inventors

Redmond, WA, United States of America

José A Blakeley