Microsoft Technology Licensing, LLC

Sumeet Priyadarshee Dash

Raghunath Ramakrishnan

Cesar Alejandro Galindo-Legaria

Arnd Christian König

Brandon Barry Haynes

Kabita Mahapatra

Dang Hai Pham

Ye Eun Park

Chi Yang

Mahadevan Sankara Subramanian

Milan Potocnik

Mohammad Shafiei Khadem

Kevin Bocksrocker

Arnd Christian KÖNIG

A hypergraph workload manager in a server is configured for failure tolerant and explainable state machine driven hypergraph execution. The hypergraph executor comprises a query optimizer, a hypergraph enlister, a pipeline analyzer, and a state machine generator. The query optimizer translates a user query into a query operator graph. The hypergraph enlister enlists the query operator graph into a hypergraph containing a set of query operator graphs representative of already submitted user queries. The enlistment is configured to join query operator graphs where it makes sense to optimize query executions. Updates to the hypergraph based on the enlistment results in a set of disconnected graphs. The pipeline analyzer performs an analysis of all operators of all queries in the hypergraph to find an optimal sequencing of execution. The state machine generator is configured to generate a hierarchical state machine for all operators of a disconnected graph of the hypergraph.

Failure tolerant and explainable state machine driven hypergraph execution

Systems and methods are provided that introduce an approach for executing a multi-query workload that leverages live execution feedback from nodes to detect resourcing issues and anomalies, and deploy real-time corrective measures for the multi-query workload. Leveraging live execution feedback from the nodes as the queries are executing make it possible to detect various resourcing issues and anomalies, and enable the system to perform corrective actions 'live' or in “real-time” during an execution of a query, and more specifically during execution of the tasks within a query.

Dynamic workload management optimizations using real-time execution feedback

Methods for distributed histogram computation in a framework utilizing data stream sketches and samples are performed by systems and devices. Distributions of large data sets are scanned once and processed by a computing pool, without sorting, to generate local sketches and value samples of each distribution. The local sketches and samples are utilized to construct local histograms on which cardinality estimates are obtained for query plan generation of distributed queries against distributions. Local statistics of distributions are also merged and consolidated to construct a global histogram representative of the entire data set. The global histogram is utilized to determine a cardinality estimation for query plan generation of incoming queries against the entire data set. The addition of new data to a data set or distribution involves a scan of the new data from which new statistics are generated and then merged with existing statistics for a new global histogram.

Distributed histogram computation framework using data stream sketches and samples

Growing community of inventors

Lake Forest, CA, United States of America

Sumeet Priyadarshee Dash